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FOREWORD 


One  of  the  research  goals  of  the  Decision  Sciences  Laboratory  is 
the  development  of  design  principles  for  automated  training  subsystems 
which  could  be  built  into  future  Information  Systems.  Such  subsystems 
would  provide  Information  Systems  with  the  capability  of  training  auto¬ 
matically  their  own  operators.  To  be  able  to  design  such  a  capability 
requires  first  the  solution  of  many  conceptual  and  experimental  problems. 
This  report  explores  some  programing  concepts  for  the  training  mate¬ 
rials  in  these  systems. 

This  report  is  one  in  a  series  supporting  Task  768204,  Automated 
Training  for  Information  Systems,  under  Project  7682,  Man-Computer 
Information  Processing.  The  research  was  conducted  during  1962  to 
1964.  The  Principal  Investigator  was  Dr.  Thomas  B.  Sheridan  and  the 
Contract  Monitor  was  Dr.  Sylvia  R.  Mayer. 

Several  Bio -Dynamics'  staff  members  have  contributed  directly  to 
the  accomplishment  of  different  parts  of  this  study:  A.  R.  Johnson, 

J.  Mickunas,  and  A.  W.  Mills  to  the  experiment  described  in  Chapter  II, 
R.  Rosenberg,  H.  Chamberlin,  and  B.  C.  Duggar  to  those  described  in 
Chapters  HI  and  IV. 


PUBLICATION  REVIEW  AND  APPROVAL 


This  Technical  Documentary  Report  has  been  reviewed  and  is  approved. 
FOR  THE  COMMANDER 


Chief,  Applications  Division 
Decision  Sciences  Laboratory 


Colonel,  USAF 

JPurector,  Decision  Sciences  Laboratory 
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EXPERIMENTAL  USE  OF 


THREE  INSTRUCTION  CONCEPTS 

Abstract 

This  report  describes  three  experiments  in  which  novel  teaching  con¬ 
cepts  were  demonstrated.  These  concepts  had  been  proposed  in  previous 
reports  but  their  effectiveness  remained  to  be  verified  experimentally. 
The  results  were: 

1.  A  teaching  program  ordered  according  to  the  discovery 
principle  significantly  reduced  errors  and  performance 
time  over  that  observed  after  training  with  a  conventional 
training  manual. 

2.  Slides  projected  directly  onto  a  control  console,  together 
with  a  taped  lecture,  were  found  to  be  an  effective  method 
of  presenting  an  automated  training  program. 

3.  Graphical  logical  flow  diagrams  were  found  to  be  efficient 
instructions  for  teaching  procedures  for  performing  a 
querying-reasoning  task. 

It  was  concluded  that  these  concepts  should  be  exploited  in  training 
programs  for  operators  of  Air  Force  Information  Systems. 
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I.  INTRODUCTION 


The  experiments  described  in  this  report  were  performed  as  part  of  a 
research  program  to  develop  general  principles  for  the  design  and  use  of 
Information  Systems  for  automated  on-the-job  training.  The  experiments 
were  designed  .to  validate  new  instructional  techniques  or  principles  which 
had  been  mentioned  in  earlier  reports  of  this  series  (1,  2).  These  methods 
and  techniques  are  believed  to  have  application  for  the  training  of  operators 
of  future  Air  Force  Information  Systems. 

There  exists  a  large  body  of  literature  on  learning  theory,  but  relatively 
little  attention  has  been  directed  towards  providing  principles  directly  appli¬ 
cable  to  the  tactical  issue  of  how  best  to  teach  particular  skills  (3).  Questions 
relating  to  program  order,  frame  size,  verbal  versus  symbolic  content,  pre¬ 
sentation  techniques,  etc.,  remain  as  impediments  to  the  specification  of  sys¬ 
tem  requirements  for  inclusion  of  automated  training  capabilities.  New  me¬ 
thods  and  techniques  can  be  derived  which  are  in  accord  with  current  theory 
and  which  offer  advantages  in  ease  of  implementation  but  because  of  uncertain¬ 
ties,  experimental  validation  is  mandatory;  for  example,  the  efficiency  with 
which  they  teach  the  required  skills  or  knowledge  must  be  demonstrated. 

The  first  experiment  (Section  II)  is  an  evaluation  of  a  teaching  program 
ordered  according  to  the  "discovery11  principle.  In  this  program,  the  trainee 
is  led  to  discover  the  need  for  each  control  function  before  the  control  device 
is  introduced  to  him.  Thus,  the  trainee  invents  a  system  which  meets  the 
perceived  task  needs.  As  more  and  more  functions  are  added,  the  aboriginal 
system  progresses  through  a  "phylogenic"  evolution  until  it  reaches  the  final, 
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sophisticated  state.  The  "discovery"  or  "phylogenic"  teaching  concept 
was  described  more  fully  in  a  previous  report  (1),  and  is  believed  to  be 
particularly  useful  for  instruction  in  the  use  of  multiple  control  functions 
and  as  a  means  of  ordering  teaching  programs  whose  content  does  not 
suggest  a  logical  order. 

Section  in  contains  a  description  of  an  experiment  in  which  graphical 
logic  flow  diagrams  were  projected  directly  onto  an  abstract  representa¬ 
tion  of  a  SAGE  console.  A  taped  lecture  accompanied  the  series  of  15 
slides  and  provided  a  program  for  instructing  trainees  in  the  use  of  con¬ 
sole  control  sequences.  This  experiment  served  two  purposes:  first,  the 
demonstration  of  a  teaching  technique  using  the  operational  equipment  but 
requiring  no  modification  to  the  equipment,  and  second,  the  demonstration 
of  the  use  of  logic  flow  diagrams  as  teaching  or  user  aids. 

The  third  experiment  (Section  IV)  involved  instruction  in  querying- 
reasoning  behavior,  the  use  of  bookkeeping  aids,  and  the  substitution  of 
diagramatic  task  descriptions  for  verbal  descriptions  in  an  instructional 
program.  The  development  of  a  logic  flow  diagraming  technique  for  use 
as  a  teaching  aid  is  described  in  detail  in  another  report  (4).  The  valida¬ 
tion  of  this  technique  for  teaching  querying-reasoning  behavior  together 
with  its  use  in  the  experiment  for  teaching  sequential  use  of  console  con¬ 
trols  suggests  that  broad  areas  of  application  are  feasible. 

The  system  design  implications  of  the  three  experiments  are  dis¬ 
cussed  in  Section  V  of  this  report.  The  three  areas  studied:  (a)  teach¬ 
ing  the  functions  of  controls  and  how  to  apply  them  analytically,  (b)  teaching 
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the  sequential  use  of  controls  and  the  logical  relationships  between  se¬ 
quences  and  mission  developments,  and  (c)  teaching  querying -reasoning 
and  bookkeeping  skills,  may  overlap  in  the  tasks  required  of  future  Air 
Force  Information  Systems  operators.  Suggestions  for  combining  the 
various  techniques  in  teaching  programs  are  included  in  Section  V. 

H.  PHYLQGENIC  TEACHING  PROGRAM 

A.  Background 

The  introduction  of  a  naive  trainee  to  his  future  role  as  opera¬ 
tor  of  a  complex  system  requires  not  so  much  the  establishment  of  a  pat¬ 
tern  of  rote  responses  but  rather  the  instillation  within  him  of  the  func¬ 
tional  purposes  of  the  displays  and  controls  to  which  he  has  access. 
Therefore,  it  is  the  meaning  or  semantic  content  of  each  component  at 
his  interface  with  the  system  that  must  be  learned.  Unfortunately,  this 
information  is  often  conveyed  to  him  in  a  language  in  which  he  may  ini¬ 
tially  have  no  fluency.  It  is  postulated  that  the  elicitation  of  an  insight 
on  the  part  of  the  trainee  for  the  need  for  a  new  function,  before  that 
function  has  been  presented  or  even  described  to  him,  will  produce  the 
most  rapid,  retrievable,  useful,  and  comprehensive  learning. 

If  we  are  to  provide  a  training  program  which  will  render  the  opera¬ 
tion  of  the  system  self -teaching,  then  we  must  consider  the  problem  of 
stimulus -response  format  so  that  sequential  insights  into  its  functions 
will  be  induced.  We  shall  ignore  for  the  moment  the  language  problem 
by  allowing  a  person  to  communicate  with  the  subject  according  to  pre- 
specified  rules. 
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An  alternative  description  of  the  role  of  the  trainee  would  character¬ 
ize  him  as  the  inventor  of  the  system  as,  step  by  step,  he  calls  for  new 
component  functions  to  improve  the  capabilities  of  the  system.  In  order 
to  promote  the  most  rapid  "invention"  of  the  desired  system,  problem 
situations  are  presented  sequentially  to  be  solved  through  use  of  the  sys¬ 
tem  itself.  Note  that  the  trainee  need  not  specify  the  physical  form  of  the 
device  but  only  its  function  or  purpose.  The  desired  function  having  been 
verbalized  adequately,  an  appropriate  system  component  is  made  avail¬ 
able  to  the  trainee  for  immediate  use  and  confirmation. 

It  is  apparent  that  a  favorable  starting  point  for  training  program  de¬ 
velopment  is  to  be  found  in  the  former  evolution  of  the  system  itself.  For 

; 

this  reason,  the  approach  to  operator  training  being  described  here  has 
been  termed  the  "Phylogenic  Method"  in  that  the  inventor -trainee  causes 
the  system  to  proceed  in  its  development  through  the  various  "phyla"  in 
the  system's  evolution  (hypothetical  or  real). 

Rigid  adherence  to  the  sequence  of  actual  system  changes  is  impos¬ 
sible  since,  on  the  one  hand,  it  would  be  very  difficult  to  identify  a  pre¬ 
cise  course  of  system  evolution,  and  on  the  other,  it  is  advantageous  to 
let  the  variable  insights  of  the  individual  trainee  adapt  the  program  to  the 
order  of  development  which  is  most  appropriate  for  him.  A  more  exten¬ 
sive  discussion  of  the  theoretical  development  of  the  phylogenic  method 
may  be  found  in  reference  1. 

A  test  system  was  fabricated  for  the  comparison  of  the  phylogenic 
method  with  other,  more  standard  methods.  It  consisted  of  a  panel  of 
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controls  which  could  be  used  to  manipulate  characteristics  of  Lissajous 
figures  on  an  oscilloscope  screen  to  make  them  match  the  desired  figure 
as  presented  on  a  transparent  overlay.  The  sequence  of  overlays  pre¬ 
sented  was  the  teaching  program. 

Initially,  it  was  planned  only  to  compare  the  teaching  performance  of 
two  alternative  methods:  a  phylogenic  ordered  program,  and  an  instruc¬ 
tion  course  using  a  written  manual  and  practice  exercises.  The  results 
of  this  comparison  were  sufficiently  interesting  to  warrant  the  addition  of 
two  modified  teaching  programs:  (a)  a  phylogenic  ordered  program  in 
which  the  trainee  had  no  physical  contact  with  the  console,  and  (b)  modi¬ 
fying  the  written  instruction  manual  to  include  a  series  of  practice  exer¬ 
cises  which  precisely  duplicated  those  used  with  the  phylogenic  program. 

B.  Experimental  Design 

1 .  Equipment 

The  console  and  display  are  shown  in  Figure  1.  The  con¬ 
sole  face  was  a  flat  panel  having  24  holes,  each  showing  a  color  coded 
shaft  or  toggle  just  out  of  reach.  Color  coded  knobs  and  shaft  extensions 
were  provided  individually  when  the  designated  control  function  was  ready 
to  be  used  during  the  phylogenic  program,  or  were  already  in  place  for 
the  manual.  The  various  control  functions  were  divided  into  two  identical 
sets  of  12  each.  Each  set  of  functions  could  be  applied  to  one  of  the  two 
Lissajous  figures  which  appeared  singly  or  simultaneously  on  the  display 
screen.  The  control  functions  were: 
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Figure  1 .  Console  with  two  identical  sets  of  12  control 
functions.  Controls  are  used  to  produce  Lissajous  fig¬ 
ures  similar  to  that  shown  on  display  sc.reen  above  the 
console. 


- 
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1.  on-off  switch  for  the  particular  figure 

2.  right -left  movement 

3.  up -down  movement 

4.  amplitude  change  along  the  x-axis 

5.  amplitude  change  along  the  y-axis 

6.  phase  shift  function  common  to  both  x  and  y-axes 

7.  partial  limiting  along  y-axis 

8.  partial  limiting  along  x-axis 

9.  vertical  line  width 

10.  horizontal  line  width 

11.  vertical  mirror -image  function 

12.  horizontal  mirror -image  function 

The  display  consisted  of  a  5 -inch  CRT  on  which  one  or  two  Lissajous 
figures  could  be  created.  A  slide  holder  was  fitted  to  the  face  of  the  CRT 
so  that  3x4  inch  transparencies  could  be  placed  between  the  observer  and 
the  display  screen.  The  opacity  of  the  transparencies  was  low  and  did  not 
obscure  the  view  of  the  Lissajous  figures.  Consequently,  the  viewer  could 
compare  the  characteristics  of  the  Lissajous  figure  with  whatever  figure 
appeared  on  the  transparency,  and  by  manipulating  controls  cause  the  two 
to  coincide  (see  Figure  1). 

Beside  the  console  was  a  button  which  the  trainee  used  to  signal  com¬ 
pletion  of  each  slide.  If  the  match  between  slide  and  Lissajous  figure(s) 
was  acceptable,  a  green  light  over  the  display  went  on.  If  not,  a  red  light 
went  on.  The  experimenter  was  separated  from  the  subject  by  a  panel. 
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but  could  observe  both  the  display  and  the  control  console  through  a  small 
aperture. 

Control  knob  and  completion  button  use  were  recorded  on  a  33 -channel 
event  recorder.  The  event  record  served  as  the  data  source  for  subse¬ 
quent  performance  analyses. 

2 .  Subjects 

Subjects  were  28  paid  male  college  students  who  responded  to  an 
advertisement  on  the  student  aid  bulletin  board.  Subjects  were  divided 
into  7  matched  groups  on  the  basis  of  results  on  the  Revised  Minnesota 
Paper  Form  Board  Test  (MPT),  Series  AA.  This  test  was  administered 
when  subjects  were  first  contacted.  The  experimental  sessions  were  then 
scheduled  for  a  later  date  with  members  of  each  matched  group  assigned 
randomly  among  the  four  training  programs. 

3.  Procedures 

a.  Instruction  Manual  (Short) 

Subjects  worked  alone  during  the  training  period  until  they 
had  completed  going  through  the  manual  and  15  practice  slides  (the  com¬ 
plete  manual  appears  in  Appendix  A).  The  manual  described  each  function, 
showed  graphically  how  the  function  affected  a  figure,  and  included  a  se¬ 
quence  of  functions  needed  to  solve  each  of  the  15  sample  problems.  The 
console  was  initially  set  up  with  all  control  knobs  in  place  and  descriptive 
diagrams  of  the  control  function  located  next  to  each  knob. 

b.  Instruction  Manual  (Long) 

Procedures  were  the  same  as  for  the  short  program  with 
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the  instruction  manual,  except  that  80  practice  slides  were  used.  The 
practice  slides  were  the  same,  and  in  the  same  order,  as  those  used 
with  the  phylogenic  program. 
c.  Phylo  genic 

Subjects  were  seated  before  the  console  which  was  without  knobs 
or  diagrams,  and  given  a  typed  set  of  instructions  (see  Appendix  B).  The 
instructions  described  the  task  to  be  performed  (match  the  figures  on  the 
CRT  to  those  on  the  slides)  and  advised  the  subject  on  how  to  request  ne¬ 
cessary  control  functions,  how  to  install  control  knobs,  and  how  to  signal 
completion  of  a  problem.  When  the  subject  had  read  the  instructions,  the 
experimenter  exposed  the  first  slide  and  the  training  program  began.  The 
subject  had  to  analyze  the  difference  between  the  initial  and  goal  figures 
and  decide  (or  discover)  what  funqtion(s)  was  needed.  If  a  knob  for  that 
function(s)  had  not  previously  been  received,  the  subject  asked  for  the 
function  (for  example,  "I  need  to  be  able  to  move  it  to  the  left").  If  the 
request  was  appropriate  to  the  problem,  a  color  coded  knob  came  down  a 
chute  and  was  then  installed  by  the  subject.  At  the  same  time,  the  sub¬ 
ject  was  allowed  to  remove  the  mask  over  the  diagram  next  to  that  control 
knob.  If  the  subject  asked  for  a  function  which  was  not  a  part  of  the  equip¬ 
ment,  or  a  function  which  was  not  needed  at  that  time,  the  experimenter 
showed  him  a  typed  message  "Your  request  cannot  be  provided  at  this 
time.  Request  a  different  control.  "  In  the  case  when  the  subject  had 
enough  controls  to  change  the  display  figures,  but  asked  for  additional 
controls,  the  experimenter  showed  him  another  typed  message  "You  have 
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enough  controls  already.  " 

When  the  subject  finished  training  on  single  Lissajous  figure  displays 
(60  figures),  he  had  acquired  a  complete  set  of  controls  for  the  left  panel 
of  the  console.  Before  proceeding  with  double  Lissajous  figure  displays, 
the  subject  rested  for  five  minutes.  Then,  he  was  instructed  that  he  would 
now  learn  to  control  two  figures  (see  Appendix  B)  and  was  provided  with 
all  of  the  second  set  of  controls.  The  subject  then  worked  through  20  more 
practice  studies,  after  which  the  training  was  complete. 

d.  Verbal  Analysis 

Procedures  for  the  verbal  analysis  modification  of  the  phylogenic 
program  were  only  different  in  that  the  subject  had  no  physical  contact  with 
the  control  panel.  The  program  consisted  of  the  same  series  of  Lissajous 
figure  displays  as  the  phylogenic  training  program.  Instead  of  analyzing 
the  figure  displays  and  manipulating  the  controls,  the  subjects  had  to  ver¬ 
balize  their  analyses  and  tell  the  experimenter  what  functions  to  use  to 
change  the  display  figure  on  the  oscilloscope  screen.  The  console  was 
completely  concealed  from  the  subject  during  the  training  program.  In 
place  of  the  console  was  a  panel  on  which  were  taped  12  blank  3x5  inch 
cards.  Each  time  the  subject  correctly  requested  a  new  control  function, 
one  of  the  3x5  inch  cards  was  removed  from  the  panel,  exposing  a  de¬ 
scription  and  diagram  of  the  control  function.  The  control  diagrams  and 
descriptions  were  in  the  same  relative  positions  as  on  the  actual  console. 
The  instructions  for  the  verbal  analysis  program  appear  in  Appendix  C. 
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After  completing  the  training  and  before  the  test,  the  subjects  were 
shown  the  console  and  given  10  minutes  to  practice  using  the  functions 
that  they  had  previously  learned. 

e.  Test 

Subjects  from  all  four  groups  were  given  the  same  test,  consist¬ 
ing  of  11  single  figure  problems  (slides)  and  9  double  figure  problems. 

The  same  set  of  instructions  was  given  all  subjects  and  the  console  ar¬ 
rangement  was  identical  in  each  case.  All  knobs  were  in  place  and  dia¬ 
grams  of  the  control  function  appeared  next  to  each  control.  All  control 
uses  were  recorded  during  the  test. 

Performance  measures  consisted  of:  (1)  training  time,  (2)  time  to 
complete  the  test  series  (individual  slide  and  total  time),  (3)  frequency  of 
use  of  inappropriate  control,  (4)  errors  in  adjustment  of  appropriate  con¬ 
trol.  Each  test  slide  was  analyzed  and  the  necessary  control  functions 
listed.  Since  all  control  functions  were  independently  controlled  by  their 
corresponding  knobs,  each  had  only  to  be  used  once.  Successive  uses, 
after  the  first,  for  an  appropriate  control  were  classed  as  "adjustment 
errors".  As  long  as  the  subject  continued  to  manipulate  the  same  control, 
it  was  counted  as  a  single  application. 

Subjects  who  did  not  complete  at  least  the  single  figure  slides  in  their 
instruction  program  within  three  hours  were  discharged  and  not  given  the 
test. 

C.  Results 

Individual  measures  obtained  from  each  subject  are  tabulated  in 
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Appendix  D.  Complete  experimental  measures  were  obtained  from  only 
25  subjects.  Two  subjects  could  not  finish  the  verbal  analysis  training 
program  within  the  three -hour  limit,  and  one  subject  could  not  finish  the 
phylogenic  training  program.  Because  the  three  subjects  who  could  not 
cope  with  the  training  programs  had  scores  among  the  lower  third  of  those 
measured  with  the  Revised  Minnesota  Paper  Form  Board  Test,  their  per¬ 
formance  was  not  considered  sufficient  reason  for  invalidating  the  phylo¬ 
genic  teaching  concept.  With  the  long  instruction  manual  program,  the 
Spearman  rank  correlation  between  high  training  time  and  low  score  on  the 

MPT  was  significant  at  the  5  percent  level  (r  =  0.  78)  as  was  the  correla- 

s 

tion  with  test  time  (r  =  0.  74).  Short  instruction  manual  time  correlations 

s 

with  score  were  in  the  same  direction  but  were  not  statistically  significant 
(r  *  0. 16  and  0. 45,  respectively). 

Among  those  subjects  who  successfully  completed  the  instruction  pro¬ 
grams,  test  time  and  number  of  both  types  of  errors  were  smaller  after 
the  phylogenic  or  the  verbal  analysis  program  than  after  either  of  the  in¬ 
struction  manual  programs  (Table  1).  When  the  subjects  were  arranged 
into  matched  groups  of  5  subjects  each,  using  the  MPT,  the  two  phylogenic 
programs  remained  superior  (Table  2).  Analyses  of  variance  were  con¬ 
ducted  on  the  five  subject  groups  data,  and  Newman -Keuls  sequential  range 

tests  were  used  to  compare  the  differences  among  individual  means  (5). 

/ 

Differences  between  training  time  and  between  number  of  errors  in  con¬ 
trol  selection  were  found  to  be  significant  at  the  5  percent  level  (F  =  4.  4 
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Table  1 


Mean  Performance  Data 

for  All  Subjects  Who  Completed  Training  (N  =  7) 
Phylogenic  Instruction 

_ Program _  Instruction  Manual 

Verbal 


Console 

Analysis 

Short 

Long 

Training  Time  (min.  ) 

60.  6* 

75.  4** 

72.  0 

103.  0 

Testing  Time  (min. ) 

28.  0* 

26.  4** 

48.  0 

41.  0 

Errors  in  Control 
Selection 

64.  0* 

36.  2** 

119.  0 

105.  0 

Errors  in  Control 
Adjustment 

176. 0* 

75.  0** 

293.  0 

248.  0 

*One  subject  did  not  finish  training  within  3  hours  and  was  discharged; 
results  are  not  included  in  group  means. 

**Two  subjects  did  not  finish  training  within  3  hours  and  were  discharged; 
results  are  not  included  in  group  means. 

Table  2 

Mean  Performance  Data 
for  Matched  Groups  of  Subjects  (N  =  5) 

Phylogenic  Instruction 


Program _  Instruction  Manual 


Console 

Verbal 

Analysis 

Short 

Long 

Training  Time  (min. ) 

58.  5 

75.  4 

72.  7 

89.2 

Testing  Time  (min. ) 

25.  6 

26.  4 

37.2 

34.  2 

Errors  in  Control 

Selection 

58.  0 

36.  2 

123.  2 

115.4 

Errors  in  Control 

Adjustment 

120.  6 

75.  0 

291.2 

280.  8 

-  13 


1 


and  F  =  4.  3  with  3,  12  d.  f. ).  The  mean  training  time  with  the  phylogenic 
program  was  significantly  less  than  with  the  long  form  of  instruction  man¬ 
ual,  and  the  mean  number  of  errors  with  the  verbal  analysis  program  was 
significantly  less  than  with  either  version  of  the  instruction  manual. 

Analyses  of  variance  were  used  to  compare  the  results  between  the 
two  discovery  programs  (phylogenic  and  verbal  analysis).  No  statistically 
significant  differences  in  performance  were  found.  When  the  results  with 
the  two  instruction  manuals  were  compared  with  one  another  using  seven 
matched  pairs  of  subjects, only  the  difference  in  training  time  was  found  to 
be  significant  (F  =  7.  289  with  1,  6  d.  f . ).  The  results  with  the  two  discov¬ 
ery  programs  were  then  pooled  (n  =  10),  as  were  the  results  with  the  two 
instruction  manual  programs  (n  =  10),  and  a  variety  of  statistical  compari¬ 
sons  between  discovery  and  instruction  manual  programs  computed.  Dif¬ 
ferences  in  testing  time,  selection  error,  and  adjustment  error  were  all 
significant  at  the  1  percent  level  or  better,  but  training  time  did  not  differ 
significantly  between  the  two  pooled  groups. 

The  results  of  the  phylogenic  console  program  were  also  compared 
with  the  results  of  the  two  instruction  manual  programs  using  three  matched 
groups  of  six  subjects  each.  Training  time  was  found  to  be  significantly 
less  with  the  phylogenic  program  (P  less  than  0.  025)  when  the  square  root 
transforms  were  compared.  Test  time,  and  both  types  of  error  were  sig- 
nificantly  less  with  the  phylogenic  program  (P  less  than  0.  05  and  0.  025, 
respectively). 
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D.  Discussion 


The  experimental  results  indicate  a  clear  superiority  for  the  dis¬ 
covery  principle  teaching  programs  as  opposed  to  conventional  instruction 
manuals  when  given  to  subjects  having  a  good  ability  to  perceive  spatial 
relations.  Moreover,  it  is  likely  that  the  spatial  relations'  restriction 
was  due  primarily  to  the  nature  of  the  specific  task  used,  and  not  to  the 
teaching  techniques. 

Large  differences  in  both  speed  and  accuracy  of  performance  on  the 
test  task  were  observed  between  the  groups  of  subjects  who  spent  roughly 
comparable  periods  of  time  learning  by  the  several  methods.  A  number 
of  hypotheses  hav&  been  considered  to  explain  these  differences.  The 
hypothesis  which  seems  most  attractive  is  a  comparison  with  linear  and 
branched  programed  instruction.  The  instruction  manual  method  is  simi¬ 
lar  to  a  linear  program  in  that  information  is  presented  in  a  single,  pre¬ 
determined  sequence.  The  reader  may  skip  while  using  the  instruction 
manual,  but  he  has  no  external  guide  as  to  when  or  how  to  skip,  and  can 
skip  advantageously  only  after  he  had  nearly  mastered  the  material.  In 
branching  programed  instruction,  the  future  program  is  contingent  upon 
the  past  behavior  of  the  student.  At  pre-selected  points,  the  student  is 
directed  to  a  more  advanced  sub -program  which  de -emphasizes  the  parti¬ 
cular  aspects  of  the  task  over  which  he  has  already  demonstrated  his  mas¬ 
tery.  The  phylogenic  program  may  act  as  an  inherently  branching  type  of 
program  in  which  the  student  determines  the  number  and  location  of  the 
branches. 
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According  to  this  hypothesis,  as  the  student  invents  the  new  func¬ 
tions  required  by  the  more  difficult  problems,  he  continues  to  use  the 
more  elementary  functions  invented  earlier,  but  in  an  increasingly  rou¬ 
tine  and  perfunctory  way.  His  attention  is  directly  to  those  aspects  of 
the  task  which  he  does  not  understand,  and  away  from  rehearsing  what 
has  already  been  mastered.  The  phylogenic  method  may  be  likened  to  a 
program  of  instruction  with  an  indefinite  number  of  branches  occurring 
at  an  indefinite  number  of  points,  the  program  created  by  each  student 
being  uniquely  adapted  to  his  individual  requirements. 

Use  of  an  instructor  for  teaching  the  experimental  task  was  not  eval¬ 
uated.  (Recall  that  the  role  the  experimenter  played  in  the  phylogenic  ex¬ 
periment  was  that  of  a  mechanical  language  interpreter,  not  an  instructor. ) 
An  instructor  might  lead  his  students  to  "discover"  the  need  for  each  con¬ 
trol  function,  or  might  simply  review  the  material  provided  in  the  instruc¬ 
tion  manual.  The  fact  that  subjects  did  not  have  to  physically  use  the  con¬ 
trols  to  "discover"  and  learn  their  functions  was  illustrated  by  the  perfor¬ 
mance  of  the  verbal  analysis  group.  Further  research  to  determine  if  the 
discovery  principle  could  be  implemented  in  a  written  instruction  manual 
appears  to  be  warranted.  Such  an  instruction  manual  would  still  be  simi¬ 
lar  to  a  linear  program,  but  the  wording  and  format  would  be  markedly  dif¬ 
ferent  from  that  conventionally  used  in  instruction  manuals. 

HI.  PROJECTION  EXPERIMENT 
A.  Background 

Training  films  have  been  used  as  teaching  aids  for  many  years. 
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However,  the  effectiveness  of  training  films  is  limited,  they  are  expen¬ 
sive  to  produce,  and  they  lack  flexibility.  It  is  hypothesized  that  poor 
transfer  from  the  film  to  the  practice  situation  is  a  major  limitation  on 
the  effectiveness  of  training  films  for  console -type  tasks.  A  method  is 
needed  whereby  the  same,  or  better,  information  can  be  presented  to  a 
trainee  while  he  is  seated  at  an  actual  console,  and  which  requires  that 
he  make  active  responses,  provides  feedback  for  knowledge  of  results, 
and  provides  reinforcement.  The  method  should  be  amenable  to  computer 
control  and  the  presentation  materials  should  be  inexpensive  to  produce. 

A  previous  experiment  had  demonstrated  the  effectiveness  of  logic 
diagrams  as  a  teaching  and  user  aid  when  printed  on  a  console  control 
panel.  However,  for  complex  tasks,  the  number  of  intersecting  and  over¬ 
lapping  diagrams  could  prove  confusing.  It  is  hypothesized  that  most  com 
plex  console  tasks  can  be  broken  into  distinct,  separate  sequences  of  ac¬ 
tions,  each  of  which  can  be  successfully  diagrammed  on  the  console  con¬ 
trol  panel.  Photo -projection  of  these  diagrams,  one  at  a  time,  onto  the 
actual  operational  console  face  (  as  individual  frames  of  a  programmed 
course)  together  with  displays  of  contingent  or  background  information 
should  permit  the  student  to  work  through  problems  in  a  realistic  fashion. 
Active  responses  could  be  required,  knowledge  of  results  and  response 
reinforcement  provided,  and  the  program  could  be  placed  under  computer 
or  trainee  control. 

To  demonstrate  the  feasibility  of  using  such  a  technique  for  teaching 
purposes,  an  experiment  was  carried  out.  Projection  diagrams  and 
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information  were  keyed  to  a  taped  lecture  and  used  to  teach  console  oper¬ 
ators  to  perform  a  simulated  mission.  For  comparison  purposes,  another 
group  of  console  operators  received  their  training  from  an  instruction  man- 

I 

ual. 

B.  Equipment 

1 .  Console 

The  console  was  an  abstract  representation  of  an  Air  Force 
SAGE  Intercept  Director's  console.  Figure  2  shows  the  arrangement  of 
the  operator's  station.  On  the  center  panel  is  the  display,  consisting  of  a 
5x5  matrix  of  lights,  and  above  the  lights  is  a  sequence  diagram  of  all 
the  possible  display  patterns  in  the  simulated  mission.  On  the  left  panel 
is  a  number  of  pushbuttons,  surrounded  by  colored  patches.  The  right 
panel  was  not  used  in  this  experiment.  A  detailed  account  of  the  develop¬ 
ment  of  the  abstract  console  has  been  presented  elsewhere  (1). 

Display  patterns  on  the  matrix  of  lights  were  controlled  by  the  exper¬ 
imenter  by  means  of  a  remote  control  box.  Through  the  control  box,  the 
experimenter  could  select  the  order  of  the  displays  and  advance  the  dis¬ 
plays  through  the  pre-selected  sequence.  Each  display  change  was  ac¬ 
companied  by  some  very  audible  mechanical  noise  which  precluded  the 
possibility  of  a  subject  not  observing  the  display  change. 

The  console  pushbuttons  were  grouped  by  function  category  (all  radio 
controls  in  one  group,  clear  computer  buttons  in  another  group,  etc. ) 
and  order  of  button  pushing  proceeded  from  left  to  right  or  from  top  to 
bottom  across  the  panel. 
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One  or  two  word  descriptions  of  the  function  of  each  control  button 
group  and  separate  labels  for  each  button  were  lettered  on  the  console. 

Approximately  two  square  feet  of  area  on  the  console  face  was  avail¬ 
able  solely  for  the  display  of  projected  information.  In  addition,  informa¬ 
tion  and  diagrams  could  be  projected  onto  the  button  area. 

2.  Training  Materials 

A  view -graph  projector  was  used  to  project  a  series  of  slides 
onto  the  console  face.  Slides  were  constructed  by  drawing  diagrams  with 
a  grease  pencil  and  by  typing  legends  onto  transparent  acetate  sheets. 

The  unused  portions  of  the  slides  were  masked  off  and  positioning  guides 
affixed  to  the  projector.  In  most  instances,  information  was  projected 
both  onto  the  area  above  the  buttons  and  onto  the  button  area  as  well.  An 
explanatory  lecture  was  recorded  on  a  tape  recorder  and  played  back 
through  a  speaker  located  behind  the  console.  Slide  changes  were  keyed 
to  the  lecture  material. 

C.  Subjects 

Subjects  were  paid  college  students  who  responded  to  a  notice  on 
the  student  employment  bulletin  board.  Twelve  subjects  were  randomly 
assigned  to  either  of  the  two  treatment  groups. 

D.  Procedures 

1.  Projected  /Taped  Instruction 

Subjects  were  seated  before  the  console  and  told  that  they 
would  now  listen  to  a  taped  set  of  instructions.  The  instructions  described 
what  the  task  was  about,  what  was  expected  of  the  trainee,  the  nature  of 
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the  training  course,  and  the  test  which  would  follow  (see  Appendix  E  for 
a  more  detailed  description  of  the  instructions  and  course  material).  A 
slide  was  then  projected  on  the  console  and  the  taped  instructions  con¬ 
tinued.  The  slides  and  tape  directed  the  trainee  through  a  series  of  se¬ 
quences  on  the  console  (which  was  turned  off).  Thus,  nothing  happened 
when  the  trainee  pushed  the  buttons  on  the  console  during  practice  and  the 
console  display  was  blank.  Representations  of  the  live  displays  were  pro¬ 
jected  corresponding  to  each  sequence.  Additional  information  was  pre¬ 
sented  which  described  the  nature  of  each  sequence,  told  the  operator  how 
to  perform  the  sequence,  then  told  him  to  follow  the  diagram  through  the 
sequences  pushing  the  designated  buttons.  After  all  sequences  had  been 
viewed,  they  were  reviewed. 

After  completion  of  the  training  course  (which  took  24  minutes),  the 
subjects  were  given  a  ten-minute  rest.  Subjects  then  returned  to  the  con¬ 
sole  and  were  told  that  their  proficiency  would  be  tested  during  several 
simulated  intercept  missions.  The  instructions  were  to  work  as  rapidly 
as  possible,  but  without  making  errors.  The  console  was  turned  on,  the 
first  display  appeared,  and  the  subject  was  on  his  own.  If  the  button  se¬ 
quence  for  each  display  was  correctly  performed,  the  display  changed  and 
the  mission  proceeded.  If  the  subject  made  a  mistake  during  a  sequence, 
the  display  would  not  change  and  the  subject  had  to  figure  out  what  the 
correct  sequence  should  be.  He  could  try  as  many  alternative  sequences 
as  he  wished,  but  all  errors  were  recorded.  Three  simulated  missions 
were  performed.  Total  time  and  number  and  type  of  errors  were  recorded. 
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2.  Instruction  Manual 


Subjects  were  seated  before  the  console,  given  an  instruction 
manual  and  a  set  of  introductory  instructions.  The  instructions  described 
the  nature  of  the  task  and  told  the  subject  that  he  was  to  study  the  instruc¬ 
tion  manual  and  the  console.  When  the  subject  signaled  that  he  was  through 
studying,  he  was  tested  during  several  simulated  intercept  missions.  The 
instructions  and  manual  are  described  in  greater  detail  in  Appendix  F. 

Performance  measures  consisted  of  training  time,  time  to  success¬ 
fully  complete  each  of  three  simulated  intercept  missions,  and  errors. 

The  errors  were  classified  as  either  (a)  an  action  sequence  inappropriate 
for  the  display  sequence,  or  (b)  an  incorrect  action  (button)  within  a  cor¬ 
rect  sequence. 

E.  Results 

Mean  performance  data  for  each  of  the  groups  with  each  of  the 
three  test  simulated  missions  appear  in  Table  3.  With  the  exception  of 
training  time,  none  of  the  differences  between  groups  was  statistically 
significant.  When  the  individual  training  times  were  compared  to  the 
median  training  time  for  all  12  subjects  and  evaluated  by  the  Fisher  exact 
probability  test  (6),  the  training  times  with  the  slides  and  tape  were  found 
to  be  smaller  significantly  often  (P  less  than  0.  04).  One  subject  in  each 
of  the  groups  successfully  completed  all  test  missions  with  no  more  than 
one  error.  On  the  third  test  mission,  four  of  the  subjects  who  were 
trained  by  the  slides  and  tape  method  and  five  of  the  subjects  trained  by 
the  instruction  manual  method  made  no  errors. 


\ 
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Table  3 


Mean  Performance  Data 

for  Console  Test  (N  =  6) 

Training 
Time 
(min. ) 

Errors 

Display 

Sequence 

Button 

Test 

Time 

(min. 

1.  Instruction  Manual 

45.  4 

(a)  Test  Mission  I 

0.  7 

1.  3 

3.  8 

(b)  Test  Mission  II 

2.  8 

5.  7 

4.  0 

(c)  Test  Mission  HI 

0.  3 

0.  8 

1.  6 

Mean,  all  missions 
combined 

3.  8 

7.  8 

9.  5 

2.  Slides  and  Tape 

24.  0 

(a)  Test  Mission  I 

2.  5 

4.  3 

3.  6 

(b)  Test  Mission  II 

3.  0 

8.  0 

4.2 

(c)  Test  Mission  HI 

0.2 

0.  3 

1.  3 

Mean,  all  missions 
combined 

5.  7 

12.  7 

9.  1 
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F.  Discussion 


Although  films,  slides,  or  taped  voice  instruction  and  user  aid 
programs  have  been  previously  used  in  variety  of  ways,  we  believe  that 
this  experiment  demonstrates  a  novel  application  which  could  be  easily 
implemented  for  use  with  simple  equipment  consoles,  or  even  with  com¬ 
plex  computer-based  system  consoles.  Computers  have  already  been 
used  to  control  presentation  of  slides  containing  instruction  materials  (6), 
or  to  generate  displays  in  a  computer -directed  teaching  program  (7). 
Evaluation  of  results,  reinforcement,  and  selection  of  program  branches 
have  been  performed  by  a  computer  in  these  systems.  A  more  simple 
decision  structure  has  even  been  programed  into  a  tape  recorder  to  con¬ 
trol  verbal  instruction  sequences  (9).  The  Smiths  Aural  Diagnostic  In¬ 
spection  Equipment  (SADIE)  guides  the  user  through  a  check-out  routine 
on  electronic  equipment  based  on  yes-no  repeat  responses  to  taped  action- 
question  instructions.  The  decision  structure  for  SADIE  is  similar  to  that 
expressed  in  man-machine  graphical  logic  flow  diagrams  (Section  IV). 
Another  system,  the  Automated  Diagnostic  Maintenance  Information  Re¬ 
trieval  System  (ADMIRE)  uses  a  library  of  instruction  slides  to  guide 
maintenance  and  check-out  sequences  (10).  However,  integration  of  in¬ 
struction  display  with  operational  controls  has  not  been  used  in  an  auto¬ 
mated  instruction  program,  or  as  a  user  aid. 

During  the  instruction  period  with  the  slides  and  taped  lecture,  none 
of  the  subjects  was  observed  to  take  inappropriate  actions.  It  would, 
therefore,  appear  useful  to  provide  console  operators  with  a  library  of 
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action  sequence  diagrams  corresponding  to  the  more  common  and  even 
some  important  uncommon  display  conditions.  These  diagrams  could  be 
projected  directly  onto  the  operational  console  control  panel,  and  addi¬ 
tional  contextual  information  displayed  on  an  adjoining  screen.  Operator 
training  would  then  emphasize  display  interpretation  and  how  to  follow 
action  sequence  diagrams,  and  would  not  emphasize  rote  memorization 
of  action  sequences. 

In  this  experiment,  subjects  who  used  the  instruction  manual  were 
allowed  to  spend  as  little  or  as  much  time  as  they  felt  necessary  to  study 
the  manual  and  the  console.  Subjects  who  heard  the  tape  and  viewed  the 
slides  did  not  have  this  opportunity.  Although  the  instruction  manual 
group  did  spend  more  time  studying,  it  is  not  known  whether  their  per¬ 
formance  would  have  been  significantly  worse  had  they  been  restricted 
to  only  24  minutes. 

IV.  INSTRUCTION  IN  QUERYING -REASONING  BEHAVIOR 

A.  Background 

The  development  of  hypotheses,  testing  of  hypotheses,  and  the 
evaluation  of  results  in  a  decision  context  will  characterize  the  behavior 
of  some  users  of  future  Air  Force  Information  Systems.  Verbal  explana¬ 
tions  of  the  relationships  between  diverging  and  converging  sequences  of 
task  cycles  (hypothesis  development -testing-evaluation)  are  often  cumber¬ 
some.  Graphical  representation  of  the  logical  contingencies  governing 
branching  paths  may  provide  a  description  of  the  relationships  among 
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decisions  and  of  sub -goals  to  system  objectives.  However,  the  effective¬ 
ness  of  such  graphical  representations  for  teaching  purposes,  without 
accompanying  lecture  or  text,  remains  to  be  demonstrated. 

The  experiment  described  in  this  section  of  the  report  was  performed 
to  demonstrate  that  graphical  logic  flow  diagrams  can  serve  as  sufficient 
and  efficient  instructions  for  a  querying -reasoning  task.  The  symbology 
and  a  methodology  for  diagraming  man -machine  tasks  have  been  described 
in  detail  elsewhere  (4).  Graphical  logic  flow  diagrams  have  been  used  suc¬ 
cessfully  as  performance  aids  with  man-machine  tasks  characterized  by 
display  interpretation  and  contingencies  among  sequences  of  action.  Com¬ 
puter  program  flow  diagrams  have  been  used  to  guide  programers,  compu¬ 
tation  analysts,  and  even  digital  computers.  The  use  of  a  diagramatic 
symbology  as  the  teaching  program  language  for  a  querying-reasoning  type 
task  represents  an  attempt  to  show  the  breadth  of  application  for  such  a 
language. 

B.  Experimental  Design 

1.  Task  Description 

The  task  selected  for  this  experiment  was  the  "numbers 
game"  described  in  an  earlier  report  (1).  The  subject's  task  in  the  num¬ 
bers  game  was  to  determine  an  unknown  four-digit  number.  The  experi¬ 
menter  simulated  an  information  system  which  the  subject  used  in  solving 
the  problem.  The  subject  queried  the  information  system  by  selecting  a 
four -digit  test  number  frcrn  a  list  of  30  test  numbers  and  requesting  a  score 
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for  the  selected  test  number.  The  score  denoted  the  number  of  digits  in 
the  test  number  which  were  identical  in  value  and  position  to  those  of  the 
experimenter's  number.  By  a  process  of  logical  deduction  and  rigorous 
bookkeeping  of  the  results  of  each  scored  test  number,  it  was  possible  to 
determine  the  unknown  number.  The  number  of  test  numbers  (queries) 
required  varied  inversely  with  the  amount  of  information  that  the  subject 
extracted  from  each  successive  score.  Since  all  test  numbers  would  not 
produce  identical  amounts  of  information,  the  subject  had  to  select  them 
sequentially  according  to  previous  results  and  reasoning. 

2.  Subjects 

Twelve  college  students  served  as  paid  experimental  subjects. 
Subjects  were  randomly  assigned  to  either  of  two  groups,  each  of  which 
received  different  instruction  programs. 

3.  Instruction  Programs 

Two  different  methods  were  employed  to  teach  the  subjects  the 
nature  of  the  game,  optimum  strategy,  and  the  bookkeeping  procedures 
which  should  be  used.  Both  methods  were  centered  around  teaching  the 
use  of  a  "bookkeeping"  sheet  developed  in  previous  experiments  with  the 
game  (1).  This  bookkeeping  sheet  (see  Figure  3)  contained:  (1)  the  list 
of  test  numbers  (LTN),  (2)  a  Score  Register  (SR)  in  which  the  subject 
was  to  enter  numbers  with  at  least  one  correct  digit  (score  greater  than 
zero),  and  (3)  a  Possible  Digits  Register  (PDR)  in  which  the  subject  would 
cross  out  digits  which  had  been  eliminated  as  possibilities. 
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Figure  3.  Bookkeeping  sheet  for  the  numbers  game. 


One  teaching  method  was  a  typewritten  illustrated  manual  (the  com¬ 
plete  manual  is  reproduced  in  Appendix  G).  The  manual  was  developed 
after  a  complete  task  analysis  and  logic  diagraming  of  the  optimum  stra¬ 
tegy.  The  manual  was  organized  according  to  the  task  logic  flow  diagram 
and  emphasized  the  successive  decision  points  and  contingencies  affecting 
bookkeeping  procedures  and  test  number  selection. 

The  other  teaching  program  consisted  of  a  hierarchical  series  of 
logic  diagrams.  Because  it  was  felt  that  most  subjects  would  not  be  fam¬ 
iliar  with  the  symbology  used  in  the  diagrams,  a  set  of  preliminary  in¬ 
structions  were  prepared  which  explained  the  symbology  and  demonstrated 
its  use  by  describing  a  familiar  task  (dialing  a  telephone).  The  diagrams 
for  the  numbers  game  are  shown  in  Figures  4,  5,  6,  7,  and  8.  The  dia¬ 
grams  are  arranged  in  three  levels,  the  highest  level  being  the  over -all 
task  description.  The  level  2  diagrams  provide  details  of  elements  of  the 
over-all  task  description,  and  the  level  3  diagrams  show  those  of  level  2. 
Each  sub-level  diagram  was  color-coded  to  aid  in  referencing  with  the 
higher  level  element  it  explained. 

4.  Procedures 

Subjects  were  seated  at  a  desk  on  which  were  placed  the  instruc¬ 
tional  materials,  3x5  cards,  and  bookkeeping  forms.  To  the  right  of  the 
subject  was  an  opaque  screen  which  concealed  the  experimenter.  In  the 
center  of  the  screen  was  a  narrow  slot  through  which  the  subjects  passed 
cards  containing  test  numbers,  and  through  which  the  experimenter  passed 
scores  for  these  numbers. 
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Subjects  were  instructed  to  study  the  instruction  manual,  or  the  logic 
diagrams,  and  to  begin  the  game  whenever  they  felt  that  they  understood 
the  procedures.  Instruction  time  commenced  when  the  subject  began  to 
read  the  instructions  and  finished  when  he  selected  a  test  number,  wrote  it 
on  a  3  x  5  card,  and  passed  it  through  the  slot  in  the  screen.  Each  time  a 
test  number  was  submitted,  the  experimenter  scored  it,  displayed  the  score 
to  the  subject,  then  recorded  the  time,  number,  and  score.  The  game  con¬ 
tinued  until  the  subject  wrote  the  four  digits  of  the  "unknown"  on  a  card  and 
passed  it  through  the  slot.  If  he  was  correct,  the  time  was  recorded  and 
the  game  ended.  If  he  was  wrong,  the  experimenter  wrote  "no"  on  the  card, 
displayed  it  to  the  subject,  and  the  game  continued.  There  was  no  penalty, 
other  than  the  time  involved,  for  an  incorrect  guess. 

Each  subject  played  six  games,  with  a  five  minute  rest  break  between 
the  third  and  fourth  games.  The  six  unknown  numbers  were  the  same  for 
all  subjects.  If  after  working  at  least  35  minutes  and  25  queries  on  any 
game  the  subject  had  not  completed  the  unknown  four  digits,  the  game  was 
discontinued  and  a  new  one  started. 

After  all  games  were  completed,  the  sequence  of  queries  was  recon¬ 
structed  by  the  experimenter,  analyzed,  and  evaluated.  Each  test  number 
selected  was  categorized  according  to  the  potential  new  information  which 
could  be  obtained  if  the  subject  followed  an  optimal  strategy,  and  if  he 
rigorously  evaluated  each  successive  score.  The  categories  were: 

(1)  maximally  useful  query,  (2)  query  with  some  value,  but  more  appro¬ 
priate  ones  could  have  been  used,  and  (3)  zero  value,  test  number  which 
could  not  provide  any  information  not  deducible  from  previous  queries. 
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For  each  game  the  total  time,  total  number  of  queries,  and  total  number 
of  queries  in  each  category  represented  the  performance  measures. 

C.  Results 

Two  subjects,  one  in  each  instruction  group,  were  unsuccessful 
on  one  of  the  six  trials.  Both  of  these  subjects  completed  the  other  five 
trails  and  their  performance  measures  were  included  in  the  data  analysis 
computations.  Individual  measures  for  each  subject  are  reported  in  Ap¬ 
pendix  H.  Because  of  the  two  incomplete  trials,  no  mean  scores  could 
be  computed.  However,  unusually  large  scores  were  occasionally  ob¬ 
served  for  subjects  on  one  trial  and  not  on  others,  and  these  scores  would 
have  had  a  large  influence  on  the  mean.  Median  scores  were  used  for 
comparison  purposes  to  avoid  undue  emphasis  on  extreme  values  and  per¬ 
mit  inclusion  of  incomplete  trial  data.  Median  scores  for  all  performance 
measures  are  listed  in  Table  4.  Paired  medians  were  statistically  com¬ 
pared  between  the  two  groups  by  the  Mann -Whitney  U  test  (6).  Median 
total  number  of  queries  were  consistently  less  for  the  group  using  the 
logic  flow  diagrams  (P  less  than  0.  01),  but  none  of  the  other  comparisons 
showed  statistically  significant  differences. 

Median  training  time  with  the  instruction  manual  was  9.  3  minutes  and 
with  the  logic  diagrams  was  14.  5  minutes. 

D.  Discussion 

Observation  of  performance  indicated  that  those  subjects  who  used 
the  logic  diagrams  generally  worked  slowly  and  more  carefully  than  did  the 
subjects  who  read  the  instruction  manual.  Bookkeeping  sheets  were  exam- 
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Table  4 


Median  Performance  Data  for 

a  Querying-Reasoning  Task 

Task 


A 

B 

C 

D 

E 

F 

Instruction  Manual 

Group  (n  =  6) 

1 .  Total  time 

18.2 

12.2 

12.  0 

10.  0 

9.  0 

10.  4 

2.  Number  of  queries* 

15.  5 

15.  5 

16.  0 

14.  3 

13.  5 

14.  5 

a)  high  value 

9.  0 

11.  5 

9.  0 

11.  5 

9.  0 

11.  5 

b)  low  value 

3.0 

1.  0 

2.0 

3.  0 

2.  5 

1.  0 

c)  no  value 

Logic  Diagram  Group 
(n  =  6) 

3.  0 

2.  5 

3.  5 

1.  0 

1.  5 

1.  0 

1.  Total  time 

17.  5 

9.  0 

14.4 

17.  6 

10.  7 

11.  2 

2.  Numbe  r  of  queries* 

14.  5 

12.  0 

14.0 

12.  5 

12.  5 

13.0 

a)  high  value 

10.  5 

9.  0 

10.  5 

10.  5 

10.  0 

11.  5 

b)  low  value 

3.  0 

2.  0 

1.  0 

2.0 

2.  0 

2.  0 

c)  no  value 

1.  0 

0.  0 

1.  5 

2.  0 

0.0 

0.  5 

*Since  all  values  reported  are  medians,  the  sum  of  the  medians  in  the 
three  categories  will  not  necessarily  equal  the  median  total  number  of 
queries. 
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ined  for  clues  as  to  how  well  the  subject  followed  the  instructions,  and  as 
to  the  nature  of  any  errors  which  he  made.  Aberrant  performance  was 
generally  due  either  to  errors  in  the  mechanics  of  bookkeeping  or  to  fail¬ 
ure  to  use  the  bookkeeping  aids.  It  was  not  possible  to  generalize  whether 
the  techniques  for  use  of  the  bookkeeping  aids  differed  between  the  two 
groups,  or  whether  the  method  of  instruction  differentially  affected  accu¬ 
racy  and  completeness  of  application. 

The  results  of  this  experiment  may  be  compared  with  those  of  a  pre¬ 
vious  experiment  in  which  a  variety  of  training  techniques  were  compared 
(1).  Median  number  of  queries  for  the  logic  diagram  group  compares  fav¬ 
orably  with  that  achieved  by  any  of  the  other  previously  studied  groups, 
except  when  the  bookkeeping  function  was  performed  for  the  subjects  by  a 
simulated  computing  system  and  displayed  after  each  query. 

Although  the  observed  median  training  time  with  the  logic  diagrams 
was  longer  than  with  the  instruction  manual,  two  subjects  completed  study 
of  the  diagrams  in  less  time  than  did  the  fastest  subjects  with  the  manual. 
Individual  training  times  with  the  diagrams  ranged  from  4  to  53  minutes 
but  only  from  8  to  13  minutes  with  the  instruction  manual.  It  is  hypothe¬ 
sized  that  those  individuals  who  required  little  time  to  study  the  logic  flow 
diagrams  were  familiar  with  flow  diagrams  in  other  fields,  and  that  if 
these  diagrams  were  used  more  frequently  for  teaching  purposes,  individ¬ 
ual  differences  in  study  time  would  be  greatly  reduced. 
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V.  APPLICATION  CONSIDERATIONS 


The  preceding  three  sections  of  this  report  describe  experiments 
which  demonstrated  the  feasibility  of  using  three  novel  teaching  programs 
to  teach  three  different  tasks.  Although  any  one  of  the  three  teaching  pro¬ 
gram  concepts  might  be  applied  to  all  three  of  the  tasks,  the  ease  and  cost 
of  implementation  would  differ  widely. 

The  first  experiment  illustrated  principles  which  should  have  wide 
application  in  terms  of  ordering  instructional  programs.  Ordering  accord¬ 
ing  to  the  discovery  principle  need  not  be  restricted  to  instruction  in  use 
of  multiple  controls.  Programs  for  use  of  any  man-machine  system  can 
exploit  this  principle.  However,  the  economics  of  providing  processing 
of  active  responses  by  the  trainee  may  restrict  strict  adherence  to  an 
automated  phylogenic -type  program  to  use  with  sophisticated  computer- 
based  systems.  Further  study  is  needed  to  determine  to  what  extent  the 
discovery  principle  should  be  used  to  guide  human  instructors,  to  upgrade 
conventional  automated  instruction  programs,  or  even  for  ordering  train¬ 
ing  manual  material. 

The  second  experiment  demonstrated  that  inexpensive  presentation 
equipment  can  be  effectively  used  to  supplement  operational  equipment  in 
an  on-the-job  teaching  program.  The  experiment  also  illustrated  the  use 
of  logic  flow  diagrams  as  a  portion  of  a  teaching  program.  For  use  with 
computer-based  systems,  access  to  the  computer  program  would  permit 
inclusion  of  a  branching  program  based  on  response  evaluation  and  of 


primary  reinforcement  of  correct  responses.  However,  the  principle 
objective  of  the  present  experiment  was  to  show  how  an  automated  instruc¬ 
tion  program  can  be  implemented  without  modifying  operational  equipment, 
and  in  which  the  operational  equipment  serves  as  an  integral  part  of  the 
instructional  program.  Since  the  instructions,  cues,  and  diagrams  are 
projected  directly  onto  the  console,  transfer  from  one  "picture"  to  another 
(as,  for  example,  with  a  book)  is  minimized.  Frames  can  be  of  variable 
size,  active  responses  required  of  the  trainee,  and  response  evaluation 
included  in  the  slide/tape  program.  Use  of  a  projection  screen  along  the 
periphery  may  suffice  when  teaching  control  sequences,  although  such  an 
arrangement  would  be  little  different  than  use  of  an  illustrated  book.  To 
permit  the  trainee  to  advance  at  his  own  pace,  a  remote  switch  for  advanc¬ 
ing  the  slide  sequence  and  for  playing  the  tape  (a  standard  increment) 
should  be  provided.  For  rapid  programing,  frame  and  lecture  material 
may  duplicate  the  lecture -demonstration  presentation  of  a  skilled  instruc¬ 
tor.  Larger  information  content  frames  in  random  access  slide/tape  units 
may  also  be  utilized  for  review  purposes  or  as  performance  aids  during 
routine  operation  of  the  equipment.  An  occasional  user  of  an  information 
system  console  might  consult  diagrams  or  contextual  information  projected 
onto  the  console  rather  than  referring  to  an  instruction  manual. 

The  use  of  logic  flow  diagrams  as  instructional  material  was  demon¬ 
strated  in  both  the  second  and  third  experiments.  The  benefits  which  may 
be  expected  to  accrue  from  use  of  these  diagrams  will  depend  largely  on 
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the  abilities  and  past  experiences  of  the  trainees.  If  the  diagrams  gain 
wide  acceptance,  their  effectiveness  and  efficiency  as  a  language  will 
increase.  The  potential  applications  for  disigramatic  representation  of 
task  characteristics  are  believed  to  be  widespread. 
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VI.  CONCLUSIONS  AND  RECOMMENDATIONS 

The  experimental  results  reported  in  this  document  confirm  and 
expand  previously  reported  preliminary  studies  and  hypotheses.  Specific 
conclusions  derived  from  the  results,  and  recommendations  for  applica¬ 
tion  or  further  study  are  as  follows: 

1.  A  phylogenic  presentation  of  control  function  ordered  according 
to  the  discovery  principle  is  an  effective  format  for  instruction.  Imple¬ 
mentation  problems  (primarily  machine  recognition  of  language)  will 
restrict  applications  unless  conventional  means  of  guiding  and  evaluating 
the  overt  response  can  be  used.  It  is  recommended  that  the  discovery 
principle  be  further  studied  for  determining  its  potential  as  a  means  of 
ordering  written  or  verbal  (instruction  manuals  or  lectures)  instructional 
material  for  future  computer-based  information  systems.  Consideration 
should  be  given  to  ordering  self-instruction  programs  according  to  the 
discovery  principle. 

2.  Graphical  logic  flow  diagrams  can  be  used  independently  or  with 
other  material  in  instructional  programs.  Although  trainees  unfamiliar 
with  such  diagrams  may  require  additional  instruction  in  symbology, 
repeated  exposure  to  this  specialized  language  will  increase  speed  and 
comprehension  of  reading.  It  is  recommended  that  graphical  logic  flow 
diagrams  be  tried  as  teaching  aids  in  an  actual  Air  Force  training  situation. 

3.  Slide /tape  instruction  programs  can  be  used  to  provide  automated 
instruction  or  practice  for  console  operators  seated  at  operational  equip¬ 
ment.  Since  no  equipment  modifications  are  required,  it  is  recommended 
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that  such  a  program  be  developed  and  used  as  a  training  device  with  an 
existing  operational  system. 
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APPENDIX  A 


Instruction  Manual  for  Operation  of  the  Lissajous  Figures  Control  Console 

The  instruction  manual  was  modeled  after  an  Operator's  Handbook  for 
a  laboratory  oscilloscope.  The  order  in  which  control  functions  were  in¬ 
troduced  initially  followed  that  used  in  the  phylogenic  program.  However, 
pilot  experiments  revealed  that  subjects  experienced  the  most  difficulty 
with  the  elliptical  control  (knob  #1)  and  that  this  difficulty  could  be  reduced 
by  introducing  that  control  last.  The  manual  appears  in  the  following  para¬ 
graphs. 


Introduction 

This  manual  describes  functions  which  are  used  to  control  a  variety 
of  figures  on  a  display  screen.  Use  this  manual  in  the  following  sequence: 

1.  read  descriptions  for  each  control, 

2.  try  to  manipulate  these  controls  so  that  you  are  familiar 
with  their  functions, 

3.  do  the  criterion  problems  as  they  are  suggested. 

You  will  have  about  one  and  one -half  hours  to  read  this  manual.  After 
the  first  hour  or  at  a  convenient  point  in  your  reading,  you  may  take  a  10- 
minute  rest. 

When  you  finish  reading  the  entire  manual  and  do  the  criterion  slides, 
you  will  get  another  10 -minute  rest.  After  this  rest  period,  you  will  be 
given  a  test  which  will  consist  of  slides  having  single  and  double  figures. 

The  Console 

The  console  contains  knobs  and  switches  which  control  figures  on  the 
display  screen.  These  controls  are  divided  into  two  groups,  each  consist¬ 
ing  of  nine  knobs  and  three  switches.  The  first  group  (the  black  knobs  and 
the  switches  below  them)  is  located  on  the  left-hand  side  of  the  console; 
the  second  group  (the  white  knobs  and  the  switches  below  them)  is  located 
on  the  right-hand  side  of  the  console.  Each  knob  and  switch  is  numbered 
for  later  identification  and  reference. 
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Diagrams,  which  graphically  indicate  the  functions  of  the  particular 
control,  are  located  next  to  each  knob  in  group  one  (left-hand  side).  The 
functions  of  the  knobs  and  switches  on  the  right-hand  side  are  identical  to 

those  of  corresponding  knobs  in  the  first  group. 

> 

To  set  up  the  console  initially,  switches  10  and  12  should  be  down,  and 
switch  11  should  be  up  (left-hand  side). 

The  Display 

The  display  screen  may  be  divided  into  four  quadrants  by  horizontal 
(x-axis)  and  vertical  (y-axis)  lines  crossing  each  other  at  the  center.  The 
x-axis  divides  the  display  screen  into  upper  and  lower  halves. 


_ upper  half 

I  ■  j  ^  —  x-axis  divides  the  display  screen  horizontally 

\.  -  —  lower  half 


The  y-axis  divides  the  display  screen  into  left  and  right  halves. 


right  half 


y-axis  divides  the  display  screen  vertically 


The  x-axis  also  represents  the  horizontal  line,  while  the  y-axis  rep¬ 
resents  the  vertical  line--these  designations  are  used  interchangeably  in 
the  descriptions  that  follow. 

Functions  of  the  Controls 
Knob  #7  -  x-axis  or  horizontal  position  control 

The  x-axis  divides  the  display  screen  into  upper  and  lower  halves. 
Knob  #7  controls  the  position  of  any  figure  along  the  x-axis.  Turning  the 
knob  clockwise,  the  figure  moves  to  the  right;  turning  the  knob  counter¬ 
clockwise,  the  figure  moves  to  the  left. 
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Display 

Screen 


Figure 


Figure  moves 
from  left  to  right 


Figure  moves 
from  right  to  left 


Knob  #7  turns 
clockwise 


Knob  #7  turns 
counterclockwise 


Knob  #6  -  y-axis  or  vertical  position  control 

The  y-axis  divides  the  display  screen  into  right  and  left  halves.  Knob 
#6  controls  the  position  of  any  figure  along  the  y-axis.  Turning  the  knob 
clockwise,  the  figure  moves  from  bottom  to  top  of  the  display  screen; 
turning  the  knob  counterclockwise,  the  figure  moves  from  top  to  bottom. 


Display 

Screen 


Figure  moves 
from  bottom 
to  top 


Figure  moves 
from  top  to 
bottom 


Figure 


Knob  #6  turns 
clockwise 


Knob  #6  turns 
counterclockwise 


Knob  #5  -  amplitude  along  the  x-axis  or  the  horizontal  line 

Amplitude  is  defined  as  the  size  or  dimension  along  a  given  axis. 
Knob  #5  controls  the  largeness  or  smallness  of  the  figure  only  along  the 
x-axis.  For  example,  turning  the  knob  clockwise  changes  the  circle  into 
an  ellipse  without  changing  the  vertical  height. 


Display^ 

Screen 


Figure  — 


^  J  tally  to  form  an 
J  ellipse 


Circle  is  ex¬ 
panded  horizon - 


Circle  is  con¬ 
tracted  horizon¬ 
tally  to  form  an 
ellipse 


Figure 


Knob  #5  turns 
clockwise 


counterclockwise 


Knob  #5  turns 
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Knob  #4  -  amplitude  along  the  y-axis  or  the  vertical  line. 

Whereas  knob  #5  controls  largeness  or  smallness  of  a  figure  along 
the  x-axis,  knob  #4  controls  largeness  or  smallness  of  a  figure  along  the 
y-axis. 


Figure  ^ 


Circle  is  ex¬ 
panded  vertically 
to  form  an  ellipse 


Display 

Screen 


Knob  #4  turns 
clockwise 


Knob  #4  turns 
counterclockwise 


The  function  of  knob  #4  is  exactly  the  same  as  that  of  knob  #5,  except  that 
knob  #4  controls  the  amplitude  along  the  y-axis  without  affecting  any  other 
dimensions. 

Knob  #3  -  x-axis  or  horizontal  partial  limiting 

Partial  limiting  is  a  change  in  some  part  of  the  figure  without  changing 
the  remaining  characteristics  of  the  figure.  Knob  #3  controls  partial  limit¬ 
ing  along  the  x-axis.  For  example: 


Display  Screen^ 


/—After  partial  limiting 


Complete  circle  before 
partial  limiting 


Knob  #3  turns  counterclockwise 


Turning  knob  #3  counterclockwise,  the  circle  changes  to  a  semicircle 
with  a  flat  side  on  the  left.  Turning  knob  #3  still  further  clockwise,  the 
semicircle  changes  into  a  solid  vertical  line  whose  height  is  equal  to  the 
diameter  of  the  circle.  Partial  limiting  function  works  on  any  figure  re¬ 
gardless  of  its  position  in  the  display.  The  main  characteristic  of  the  par¬ 
tial  limiting  function  is  a  vertical  line  which  represents  the  reduced  part 
of  the  figure. 
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Knob  #2  -  y-axis  or  vertical  partial  limiting 

Knob  #2  controls  partial  limiting  along  the  y-axis. 


.After  partial  limiting 

Complete  circle  before 
partial  limiting 


Knob  #2  turns  counterclockwise 


The  diagram  illustrates  the  effects  of  partial  limiting  on  the  circle. 
The  remaining  characteristics  of  y-axis  partial  limiting  are  the  same  as 
those  of  x-axis  partial  limiting. 

Knob  #9  -  x-axis  or  horizontal  line  width  (high  frequency) 

To  produce  heavy  or  tube -like  figures  which  have  a  three-dimensional 
appearance,  a  high-frequency  component  is  used.  Knob  #9  controls  the 
high-frequency  component  along  the  x-axis  and  will  expand  lines  of  any 
figure  in  this  direction. 


Before  addition  of 
high  -frequency 
component 


After  addition  of 
horizontal  high  fre¬ 
quency  component 


Straight 
Lines  *— 


Before 


After 


Knob  #9  turns  clockwise 


Knob  #9  expands  the  lines  of  any  figure  along  the  x-axis  by  adding  a 
horizontal  high-frequency  component. 

Knob  #8  -  y-axis  or  vertical  line  width  (high  frequency) 

Knob  #8  controls  the  high  -frequency  component  along  the  y-axis,  and 
will  expand  lines  of  any  figure  vertically. 
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component  frequency  com¬ 

ponent 

Knob  #8  turns  clockwise 

The  high-frequency  component  can  be  used  with  any  figure  appearing  on 
the  display  screen,  but  knob  #8  controls  it  only  along  the  y-axis. 

Switch  #10  -  x-axis  or  horizontal  mirror-image 

The  console  is  not  designed  for  continuous  rotation  of  the  figures  (such 
as  triangles  being  rotated  clockwise).  Instead,  the  limited  rotation  is  pro¬ 
vided  by  rotating  the  figure  in  discreet  steps  about  x-  and  y-axes.  Switch 
#10  is  used  to  rotate  the  figure  about  x-axis;  i.  e.  ,  rotate  the  figure  around 
the  horizontal  line. 


Raising  switch  #10  flips  from 
upward  to  downward  position. 


Switch  #10  can  be  used  to  obtain  mirror  -images  of  all  figures,  but  the  end 
result  is  best  seen  with  noncircle  figures. 

Switch  #12  -  y-axis  or  vertical  mirror-image 

The  function  of  switch  #12  is  to  rotate  any  figure  about  y-axis  or  verti¬ 
cal  line. 
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Display  Screen - 


—  Figure  before  rotation 


Raising  switch  #12  flips  from 
right  to  left 


Again,  the  best  figures  for  vertical  mirror -image  are  noncircles. 


Knob  #1  -  common  to  both  x-  and  y-axes 


Knob  #1  is  used  to  produce  elliptical  figures  or  lines  which  are  inclined 
at  45°  (this  angle  will  vary  if  the  vertical  and  horizontal  gains,  knobs  #4  and 
#5,  are  unequally  set).  Knob  #1  can  be  used  with  most  figures,  but  its  ef¬ 
fects  will  be  most  noticeable  on  circles,  semicircles,  and  ellipses.  Knob 
#1  will  have  no  effect  on  figures  which  have  partial  limiting  functions  in¬ 
creased  to  their  maximum. 


Display 

Screen 


Circle 


*© 


o 


Knob  #1  turns 
counterclockwise 

Turning  knob  #1  changes  a  circle  into  an  ellipse  which  is  inclined  at  45°. 
This  angle  is  obtained  if  both  vertical  and  horizontal  gains  are  set  equally. 
Turning  knob  #1  still  further,  ellipse  is  reduced  into  a  straight  line,  which  is 
inclined  at  45°. 


Available  Figures 

A  great  variety  of  figures  can  be  reproduced  on  the  display  by  manipulat¬ 
ing  the  controls  individually  or  in  various  combinations.  All  figures  may  be 
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regarded  as  distortions  of  a  circle,  produced  through  changes  in  amplifi¬ 
cation,  roundness,  limiting,  addition  of  high  -frequency  components  to 
broaden  the  lines,  or  rotation.  Starting  with  the  basic  circle,  it  should 
be  possible  to  analyze  which  controls  will  be  required,  and  the  order  in 
which  they  should  be  adjusted. 

Presentation  of  Criterion  Slides 


Translucent  slides  illustrating  the  figures  which  you  must  reproduce 
can  be  inserted  in  front  of  the  display  screen.  You  will  be  able  to  see 
simultaneously  the  figure  on  the  display  screen  and  the  figure  on  the  trans¬ 
lucent  slide.  Match  reasonably  close  the  figure  on  the  display  screen  in 
size,  shape,  and  position  to  the  slide  figure.  To  insert  the  slide,  orient 
it  so  that  the  number  is  inverted  and  appears  in  the  upper  left-hand  corner, 
and  the  cross-hatched  thumb  mark  appears  in  the  lower  left-hand  corner 
facing  you.  Insert  slides  into  the  receptacle  which  is  just  below  the  dis¬ 
play  screen.  There  is  a  catch  on  the  left  of  the  receptacle  to  help  you  in 
the  positioning  of  the  slide.  Check  how  the  slide  figure  differs  from  that 
already  on  the  display  screen.  Decide  which  controls  will  be  used,  and 
the  order  in  which  they  should  be  manipulated.  Then  change  the  figure  on 
the  display  screen  to  match  that  on  the  slide.  The  order  of  slides  is  from 
one  to  sixty  for  single  figures  (using  left-hand  group  of  console  controls), 
and  60  to  80  for  double  figures. 

There  is  a  listing  of  controls  for  each  slide  on  3  x  5  inch  cards.  The 
order  of  control  manipulations  is  suggestive  in  nature  and  should  be  used 


as  a  guide.  The  list  of  controls  for  slide  #1  is  applicable  only  if  the  fig¬ 
ure  on  the  display  screen  is  a  circle.  The  list  of  controls  for  slide  #2  is 
applicable  only  if  the  figure  on  the  display  screen  is  similar  to  that  on  the 
previous  slide.  Similar  assumptions  are  true  for  the  remaining  slides. 

Slide  #1  -  Controls: 

#7  Adjust  horizontal  position 

Slide  #2  -  Controls: 

#6  Adjust  vertical  position 

Slide  #3  -  Controls: 

#6  Adjust  vertical  position 
#7  Adjust  horizontal  position 

Slide  #4  -  Controls: 

#1  Make  an  ellipse 
#6  Adjust  vertical  position 
#7  Adjust  horizontal  position 

Slide  #5  -  Controls: 

#1  Make  a  single  line 
#6  Adjust  vertical  position 
#7  Adjust  horizontal  position 
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Slide  #6  -  Controls: 

Slide  #7  -  Controls: 
Slide  #8  -  Controls: 

Slide  #9  -  Controls: 

Slide  #10  -  Controls: 

Slide  #11  -  Controls: 

Slide  #12  -  Controls: 

Slide  #13  -  Controls: 
Slide  #14  -  Controls: 

Slide  #15  -  Controls: 

Slide  #16  -  Controls: 


Slide  #17  -  Controls: 


Slide  #18  -  Controls: 


Slide  #19  -  Controls: 


#1  Make  a  circle 
#6  Adjust  vertical  position 

#5  Increase  horizontal  gain 

#4  Increase  vertical  gain 
#6  Adjust  vertical  position 
#7  Adjust  horizontal  position 

#4  Reduce  vertical  gain 
#5  Reduce  horizontal  gain 
#7  Adjust  horizontal  position 

#5  Reduce  horizontal  gain 
#7  Adjust  horizontal  position 

#5  Increase  horizontal  gain 
#6  Adjust  vertical  gain 
#4  Reduce  vertical  gain 

#7  Adjust  horizontal  position 
#6  Adjust  vertical  position 
#4  Increase  vertical  gain 

#1  Make  an  ellipse 

#1  Make  a  circle 
#5  Reduce  horizontal  gain 

#1  Make  an  ellipse 
#4  Reduce  vertical  gain 

#1  Make  a  circle 
#4  Increase  vertical  gain 
#5  Increase  horizontal  gain 

#1  Make  a  single  line 
#4  Reduce  vertical  gain 
#6  Adjust  vertical  position 

#5  Reduce  horizontal  gain 
#7  Adjust  horizontal  position 
#6  Adjust  vertical  position 

#1  Make  a  circle 
#4  Increase  vertical  gain 
#5  Increase  horizontal  gain 
#7  Adjust  horizontal  position 
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Slide  #20  -  Controls: 

Slide  #21  -  Controls: 

Slide  #22  -  Controls: 
Slide  #23  -  Controls: 

Slide  #24  -  Controls: 

Slide  #25  -  Controls: 

Slide  #26  -  Controls: 


Slide  #27  -  Controls: 
Slide  #28  -  Controls: 

Slide  #29  -  Controls: 
Slide  #30  -  Controls: 

Slide  #31  -  Controls: 

Slide  #32  -  Controls: 


#6  Adjust  vertical  position 
#7  Adjust  horizontal  position 

#3  Increase  horizontal  partial  limiting 
#7  Adjust  horizontal  position 
#6  Adjust  vertical  position 

#2  Increase  vertical  partial  limiting 

#2  Reduce  vertical  partial  limiting 
#3  Reduce  horizontal  partial  limiting 

#4  Reduce  vertical  gain 

#3  Increase  horizontal  partial  limiting 

#3  Reduce  horizontal  partial  limiting 
#5  Reduce  horizontal  gain 
#2  Increase  vertical  partial  limiting 

#2  Reduce  vertical  partial  limiting 
#4  Increase  vertical  gain 
#5  Increase  horizontal  gain 
#6  Adjust  vertical  position 
#7  Adjust  horizontal  position 

#1  Make  an  ellipse 

#1  Make  a  larger  ellipse 

#3  Increase  horizontal  partial  limiting 

#2  Increase  vertical  partial  limiting 

#2  Reduce  vertical  partial  limiting 
#3  Reduce  horizontal  partial  limiting 

#1  Make  a  single  line 

#3  Increase  horizontal  partial  limiting 

#4  Increase  vertical  gain 

#2  Increase  vertical  partial  limiting 
#5  Increase  horizontal  gain 
#7  Adjust  horizontal  position 
#1  Make  an  ellipse 
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Slide  #33  -  Controls: 


Slide  #34  -  Controls: 

Slide  #35  -  Controls: 
Slide  #36  -  Controls: 
Slide  #37  -  Controls: 

Slide  #38  -  Controls: 

Slide  #39  -  Controls: 


S  lide  #40  -  Controls: 


Slide  #41  -  Controls: 

Slide  #42  -  Controls: 


Slide  #43  -  Controls: 


Slide  #44  -  Controls: 


#2  Reduce  vertical  partial  limiting 
#3  Reduce  horizontal  partial  limiting 
#1  Make  a  circle 
#4  Reduce  vertical  gain 
#5  Reduce  horizontal  gain 

#4  Reduce  vertical  gain 
#5  Reduce  horizontal  gain 
#7  Adjust  horizontal  position 

#9  Increase  horizontal  high-frequency  component 

#8  Increase  vertical  high-frequency  component 

#8  Reduce  vertical  high-frequency  component 
#9  Reduce  horizontal  high-frequency  component 

#3  Increase  horizontal  partial  limiting 
#9  Increase  horizontal  high-frequency  component 

#2  Increase  vertical  partial  limiting 
#9  Reduce  horizontal  high-frequency  component 
#8  Increase  vertical  high-frequency  component 

#8  Reduce  vertical  high-frequency  component 
#2  Reduce  vertical  partial  limiting 
#3  Reduce  horizontal  partial  limiting 
#7  Adjust  horizontal  position 

#1  Make  an  ellipse 

#9  Increase  horizontal  high-frequency  component 
#6  Adjust  vertical  position 

#1  Make  a  circle 

#8  Increase  vertical  high-frequency  component 
#6  Adjust  vertical  position 

#8  Reduce  vertical  high-frequency  component 
#9  Reduce  horizontal  high-frequency  component 
#1  Make  an  ellipse 

#1  Make  a  solid  line 

#2  Increase  vertical  partial  limiting 

#9  Increase  horizontal  high-frequency  component 
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Slide  #45  -  Controls: 

Slide  #46  -  Controls: 

Slide  #47  -  Controls: 
Slide  #48  -  Controls: 
Slide  #49  -  Controls: 

Slide  #50  -  Controls: 

Slide  #51  -  Controls: 


Slide  #52  -  Controls: 

Slide  #53  -  Controls: 

Slide  #54  -  Controls: 


Slide  #55  -  Controls: 


Slide  #56  -  Controls: 


#3  Increase  horizontal  partial  limiting 
#2  Make  an  ellipse 

#8  Increase  vertical  high-frequency  component 

#8  Reduce  vertical  high-frequency  component 
#9  Reduce  horizontal  high-frequency  component 

#10  Mirror -image  about  horizontal  axis 

#12  Mirror-image  about  vertical  axis 

#10  Mirror -image  about  horizontal  axis 
#12  Mirror -image  about  vertical  axis 

#10  Mirror -image  about  horizontal  axis 
#9  Increase  horizontal  high-frequency  component 

#12  Mirror -image  about  vertical  axis 
#10  Mirror-image  about  horizontal  axis 
#8  Increase  vertical  high-frequency  component 
#6  Adjust  vertical  position 
#7  Adjust  horizontal  position 

#8  Reduce  vertical  high-frequency  component 
#9  Reduce  horizontal  high-frequency  component 
#12  Mirror -image  about  vertical  axis 

#10  Mirror -image  about  horizontal  axis 
#1  Adjust  ellipse 

#12  Mirror -image  about  vertical  axis 
#1  Adjust  ellipse 
#6  Adjust  vertical  position 
#7  Adjust  horizontal  position 

#10  Mirror -image  about  horizontal  axis 
#12  Mirror -image  about  vertical  axis 
#1  Adjust  ellipse 

#1  Make  a  single  line 

#10  Mirror -image  about  horizontal  axis 

#8  Increase  vertical  high-frequency  component 


Slide  #57  -  Controls:  #1  Make  an  ellipse 

i  #12  Mirror -image  about  vertical  axis 

#9  Increase  horizontal  high-frequency  component 
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Slide  #58  -  Controls:  #1  Make  a  single  line 

#10  Mirror -image  about  horizontal  axis 
#12  Mirror-image  about  vertical  axis 


Slide  #59  -  Controls:  #8  Reduce  vertical  high-frequency  component 

#9  Reduce  horizontal  high-frequency  component 
#1  Make  an  ellipse 


Slide  #60  -  Controls:  #2  Reduce  vertical  partial  limiting 

#3  Reduce  horizontal  partial  limiting 
#1  Make  a  circle 
#4  Reduce  vertical  gain 
#5  Reduce  horizontal  gain 


Double  Figures 

Two  figures  can  be  simultaneously  displayed  by  using  both  the  left- 
and  the  right-hand  groups  of  controls.  The  knobs  and  switches  in  the 
right-hand  group  are  numbered  to  correspond  to  those  in  the  left-hand 
group.  It  is  advisable  to  work  on  reproducing  one  figure  at  a  time,'  then 
switching  to  the  other  figure  when  the  first  is  completed.  When  switch 
#11  is  in  the  up  (ON)  position,  the  figure  produced  by  the  left-hand  group 
of  controls  will  appear  on  the  display  screen.  When  switch  #13  is  in  the 
up  (ON)  position,  the  figure  produced  by  the  right-hand  group  of  controls 
will  appear  on  the  display  screen.  If  both  switches  are  in  the  ON  posi¬ 
tion,  both  figures  will  appear  simultaneously  on  the  display  screen. 

Presentation  of  Criterion  Slides 

(Double  Figures) 

The  order  of  slides  for  double  figures  (both  groups  of  controls  are 
needed)  is  from  61  to  80. 


The  listing  of  controls  and  their  order  are  suggestive  in  nature.  It 
is  assumed  that  Figure  A  is  controlled  by  the  left  side  of  controls,  and 
Figure  B  by  the  right  side  of  controls.  Moreover,  it  is  assumed  that  the 
figures  on  the  display  screen  are  similar  to  those  on  the  preceding  slide. 

Slide  #61  -  Figure  A.  Controls:  #6  Adjust  vertical  position 

#7  Adjust  horizontal  position 
Figure  B.  Controls:  #4  Adjust  vertical  gain 

#5  Adjust  horizontal  gain 
#6  Adjust  vertical  position 
#7  Adjust  horizontal  position 
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Slide  #62  -  Figure  A.  Controls: 

Figure  B.  Controls: 

Slide  #63  -  Figure  A.  Controls: 

Figure  B.  Controls: 


Slide  #64  -  Figure  A.  Controls: 


Figure  B.  Controls: 

Slide  #65  -  Figure  A.  Controls: 

Figure  B.  Controls: 

Slide  #66  -  Figure  A.  Controls: 

Figure  B.  Controls: 

Slide  #67  -  Figure  A.  Controls: 

Figure  B.  Controls: 


#2  Increase  vertical  partial  limiting 
#6  Adjust  vertical  position 
#4  Increase  vertical  gain 
#5  Reduce  horizontal  gain 

#10  Mirror -image  about  horizontal  axis 
#6  Adjust  vertical  position 
#5  Increase  horizontal  gain 
#2  Increase  vertical  partial  limiting 
#3  Increase  horizontal  partial  limiting 
#6  Adjust  vertical  position 
#7  Adjust  horizontal  position 

#3  Increase  horizontal  partial  limiting 

#1  Make  a  single  line 

#12  Mirror -image  about  vertical  axis 

#10  Mirror -image  about  horizontal  axis 

#6  Adjust  vertical  position 

#7  Adjust  horizontal  position 

#10  Mirror -image  about  horizontal  axis 

#6  Adjust  vertical  position 

#7  Adjust  horizontal  position 

#1  Make  an  ellipse 

#9  Increase  horizontal  high-frequency 
component 

#7  Adjust  horizontal  position 
#9  Increase  horizontal  high  frequency 
component 

#1  Make  a  solid  line 
#8  Increase  vertical  high-frequency 
component 

#7  Adjust  horizontal  position 
#8  Increase  vertical  high -frequency 
component 

#1  Make  an  ellipse 

#12  Mirror-image  about  vertical  axis 
#12  Mirror -image  about  vertical  axis 
#7  Adjust  horizontal  position 


Slide  #68  -  Figure  A.  Controls:  #8  Reduce  vertical  high-frequency 

component 

#9  Reduce  horizontal  high-frequency 
component 

Figure  B.  Controls:  #8  Reduce  vertical  high-frequency 

component 

Figure  B.  Controls:  #8  Reduce  horigontalhigh  -frequency 

component 
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Slide  #69  -  Figure  A. 

Figure  B. 


Slide  #70  -  Figure  A. 

Figure  B. 


Slide  #71  -  Figure  A. 

Figure  B. 


Slide  #72  -  Figure  A. 


Figure  B. 
Slide  #73  -  Figure  A. 


Figure  B. 


Slide  #74  -  Figure  A. 

Figure  B. 


Controls: 

Controls: 


Controls: 

Controls: 


Controls: 

Controls: 


Controls: 


Controls: 

Controls: 


Controls: 


Controls: 

Controls: 


#1  Make  a  single  line 
#1  Make  a  single  line 
#8  Increase  vertical  high  frequency 
component 

#9  Increase  horizontal  high-frequency 
component 

#2  Reduce  vertical  partial  limiting 
#2  Reduce  vertical  partial  limiting 
#3  Reduce  horizontal  partial  limiting 
#12  Mirror -image  about  vertical  axis 

#7  Adjust  horizontal  position 
#8  Reduce  vertical  high-frequency 
component 

#9  Reduce  horizontal  high-frequency 
component 

#3  Increase  horizontal  partial  limiting 
#10  Mirror-image  about  horizontal  axis 
#12  Mirror -image  about  vertical  axis 

#10  Mirror-image  about  horizontal  axis 
#6  Adjust  vertical  position 
#7  Adjust  horizontal  position 
#7  Adjust  horizontal  position 

#3  Reduce  horizontal  partial  limiting 
#2  Increase  vertical  partial  limiting 
#8  Increase  vertical  high-frequency 
component 

#7  Adjust  horizontal  position 

#3  Increase  horizontal  partial  limiting 

#4  Increase  vertical  gain 

#6  Adjust  vertical  position 

#9  Increase  horizontal  high-frequency 

component 

#6  Adjust  vertical  position 
#3  Reduce  horizontal  partial  limiting 
#2  Increase  vertical  partial  limiting 
#8  Increase  vertical  high-frequency 
component 

#6  Adjust  vertical  position 
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Slide  #75  -  Figure  A.  Controls:  #9  Increase  horizontal  high-frequency 

component 

Figure  B.  Controls:  #2  Reduce  vertical  partial  limiting 

#3  Increase  horizontal  partial  limiting 
#4  Reduce  vertical  gain 
#6  Adjust  vertical  position 


Slide  #76 


-  Figure  A.  Controls:  #2  Reduce  vertical  partial  limiting 
Figure  B.  Controls:  #3  Reduce  horizontal  partial  limiting 

#12  Mirror -image  about  vertical  axis 


Slide  #77  -  Figure  A.  Controls:  #8  Reduce  vertical  high-frequency 

component 

#9  Reduce  horizontal  high-frequency 
component 

Figure  B.  Controls:  #9  Reduce  horizontal  high-frequency 

component 


Slide  #78  -  Figure  A.  Controls:  #6  Adjust  vertical  position 

#7  Adjust  horizontal  position 
Figure  B.  Controls:  #8  Reduce  vertical  high-frequency 

component 

#6  Adjust  vertical  position 
#7  Adjust  horizontal  position 


Slide  #79  -  Figure  A.  Controls:  #1  Make  a  circle 

#6  Adjust  vertical  position 
#7  Adjust  horizontal  position 
#9  Increase  horizontal  frequency 
Figure  B.  Controls:  #6  Adjust  vertical  position 

#7  Adjust  horizontal  position 


Slide  #80  -  Figure  A.  Controls:  #9  Reduce  horizontal  high-frequency 

component 

Figure  B.  Controls:  #1  Make  a  circle 

#4  Reduce  vertical  gain 
#5  Reduce  horizontal  gain 
#6  Adjust  vertical  position 
#7  Adjust  horizontal  position 
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APPENDIX  B 


Instructions  for  the  Phylogenic  Teaching  Program 

The  instructions  were  presented  in  two  parts:  the  first  described 
the  task  and  procedures,  and  was  given  to  the  subject  as  soon  as  he  was 
seated,  the  second  part,  (additional  instructions)  was  given  to  the  subject 
after  he  had  completed  all  the  single  figure  slides. 

Instructions 


Your  task  is  to  learn  the  functions  which  control  the  figure  on  the  dis¬ 
play  screen. 

In  front  of  you  is  a  console  to  control  figures  on  the  screen.  The 
console  consists  of  two  panels  of  controls --the  left  panel  and  the  right 
panel.  You  will  use  the  left  panel  first.  There  are  12  numbered  places 
for  controls  (9  knobs  and  3  switches). 

Learn  to  control  the  figures  on  the  screen  by  changing  them  to  match 
reasonably  well  the  figures  on  the  slides  which  will  be  inserted  in  front  of 
the  screen.  You  will  have  to  decide  how  these  figures  differ  from  one 
another,  and  what  control(s)  you  need  to  change  the  figures  on  the  screen. 

Initially  the  left  panel  will  not  have  any  controls.  When  you  need  a 
new  control,  tell  me  in  your  own  words  what  such  a  control  will  do  to  the 
screen.  If  your  requested  control  is  actually  needed  to  change  the  figure 
at  that  time,  I  will  give  it  to  you.  A  numbered  knob  or  a  switch  will 
appear  on  the  table  to  the  right  of  the  console.  Take  that  numbered  con¬ 
trol  and  match  it  to  the  number  on  the  panel.  Before  inserting  the  control 
into  its  proper  place,  remove  the.  numbered  label  and  place  it  on  the  upper 
part  of  the  console.  Under  the  numbered  label  is  a  graphical  drawing 
which  shows  what  the  control  does.  You  may  not  look  under  labels  until 
you  receive  controls  for  those  positions. 

If  your  request  for  a  control  is  untimely,  or  if  you  request  a  control 
to  change  too  many  features  of  the  figure  on  the  screen,  or  change  it  in  an 
unacceptable  manner,  the  request  will  not  be  granted.  I  will  show  you  a 
comment  to  this  effect. 

When  you  have  matched  reasonably  close  the  figure  on  the  screen  to 
that  on  the  slide,  press  the  button  on  the  right  side  of  the  console. 


-  59  - 


If  the  match  is  close  enough,  you  will  see  a  green  light  (above  the 
display  screen)  go  on.  Then,  we  will  proceed  to  the  next  slide.  If  your 
match  is  not  adequate,  a  red  light  will  go  on.  Then  you  will  have  to 
readjust  your  controls  so  that  both  figures  match  reasonably  well. 

Additional  Instructions 

You  learned  to  control  single  figures  on  the  display  screen.  Now 
you  will  learn  to  control  two  figures  on  the  screen. 

The  controls  on  the  right  hand  side  panel  are  identical  in  functions 
to  those  on  the  left  hand  side.  Insert  the  white  knobs  and  switches  into 
the  right  hand  side  panel.  Be  certain  that  these  white  controls  are 
matched  by  number  and  place  to  the  controls  on  the  left  hand  side. 

It  is  advisable  to  work  on  one  figure  at  a  time.  When  you  have  a  rea¬ 
sonably  close  matching  of  both  single  figures,  turn  on  both  panels  to  dis¬ 
play  both  figures  simultaneously. 


APPENDIX  C 


Instructions  for  the  Verbal  Analysis  Teaching  Program 

As  with  the  phylogenic  program, instructions  were  presented  at  the 
start,  and  then  additional  instructions  were  given  after  all  the  single 
figure  slides  had  been  completed. 

Instructions 


Your  task  is  to  learn  to  analyze  the  differences  between  two  geomet¬ 
rical  figures,  and  then  specify  how  to  change  one  of  them  so  that  they  are 
identical.  To  accomplish  these  changes  you  will  be  restricted  to  eleven 
control  "functions".  Thus,  your  task  is  also  to  learn  these  functions. 

For  each  problem  there  will  be  two  geometrical  figures,  one  on  the 
display  screen,  and  the  other  on  a  transparent  slide  which  will  be  placed 
in  front  of  the  display  screen.  When  the  slide  is  first  inserted,  the  two 
figures  will  not  match.  You  will  decide  how  these  two  figures  differ  from 
one  another,  and  what  "functions"  you  will  need  to  change  the  figure  on 
the  display  screen  to  match  that  on  the  slide  (for  example,  if  the  two  fig¬ 
ures  are  geometrically  the  same,  but  that  on  the  slide  is  above  the  other, 
you  would  wish  to  raise  the  display  screen  figure.  Thus  you  would  use  a 
"vertical  position"  or  y-axis  function).  You  will  not  be  told  beforehand 
what  the  available  "functions"  are,  but  as  you  ask  for  needed  functions 
your  requests  will  be  evaluated  and  if  it  is  appropriate  the  figure  will  be 
altered  and  you  will  be  instructed  to  remove  the  covering  from  one  of  the 
cards  on  the  board  in  front  of  you.  The  function  which  you  requested  is 
described  under  this  covering  and  may  be  requested  on  subsequent  prob¬ 
lems  whenever  you  think  it  appropriate.  If,  however,  the  function  which 
you  initially  request  is  not  one  of  the  eleven  control  functions  you  will 
receive  a  "comment".  You  may  not  look  under  the  numbered  cards  on 
the  board  until  you  are  told  to  do  so. 

When  you  are  satisfied  that  the  two  figures  correspond  in  size,  shape, 
and  position,  say  that  you  are  "ready".  If  the  match  is  satisfactory  you 
will  see  a  green  light  over  the  screen  and  you  may  go  on  to  the  next  prob¬ 
lem.  If  the  match  is  net  satisfactory  the  red  light  will  come  on  and  you 
will  have  to  continue  to  improve  the  match. 
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Additional  Instructions 

You  have  learned  to  match  single  figures  on  the  display  screen.  Now 
you  will  learn  to  match  double  figures. 

Match  one  of  the  controlled  figures  at  a  time  to  one  on  the  slide.  Iden 
tify  the  figure  you  wish  to  control  by  referring  to  the  right  or  left  hand  fig 
ure.  When  both  figures  match  the  two  on  the  slide  signal  that  you  are 
"ready". 


APPENDIX  D 


Individual  Performance  Measures  for  the  Lissajous  Figure 

Control  Console  Experiment 


Training 

Test 

Control  Errors 

MPT 

Score 

Time 
(min. ) 

Time 
(min.  ) 

Selection 

Adjustment 

Phylogenic 

Console 

1 

29 

❖ 

* 

•JU 

-i* 

A, 

2 

32 

72.  5 

22.  4 

66 

90 

3 

48 

56.  5 

30.  0 

43 

94 

4 

49 

71.  0 

42.  1 

94 

275 

5 

50 

62.3 

24.2 

66 

206 

6 

57 

52.  5 

24.  7 

52 

57 

7 

57 

48.8 

46.  5 

63 

156 

Verbal 

Analysis 

1 

33 

* 

JU 

«T* 

❖ 

2 

32 

75.  0 

20.  8 

15 

16 

3 

43 

<JU 

■v* 

❖ 

* 

* 

4 

48 

65.  0 

20.  4 

21 

101 

5 

53 

89.  0 

41.8 

64 

129 

6 

54 

76.  0 

26.  7 

46 

66 

7 

59 

72.0 

22.  1 

35 

63 

Instruction 

Manual,  Shor 
1 

; 

28 

75 

62.  8 

140 

398 

2 

35 

90 

56.  5 

138 

406 

3 

44 

60 

38.  7 

77 

200 

4 

47 

60 

25.  0 

143 

333 

5 

50 

40 

19.  6 

70 

102 

6 

58 

89 

41.  0 

154 

382 

7 

60 

90 

43.  9 

111 

233 

Instruction 
Manual,  Long 
1 

r 

29 

155 

57.  6 

68 

150 

2 

38 

104 

47.  8 

95 

289 

3 

45 

118 

61. 9 

89 

179 

4 

49 

86 

48.  7 

251 

72  5 

5 

52 

77 

29.  0 

103 

234 

6 

52 

85 

14.  1 

55 

44 

7 

60 

94 

31.2 

73 

112 

*Did  not  complete  training  within  three  hours. 
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APPENDIX  E 


Instructions  for  Slide  and  Tape  SAGE  Console  Task 

Teaching  Program 

As  soon  as  the  subject  was  seated  before  the  console  the  taped  intro¬ 
ductory  instructions  commenced.  The  instructions  described  the  task 
which  was  to  be  learned,  the  manner  of  presentation,  and  the  main  parts 
of  the  console.  Slides  were  projected  onto  the  console  just  above  the  con¬ 
trol  panel  to  illustrate  the  taped  material.  Six  slides  were  used  in  the 
introductory  material:  one  which  described  the  various  console  parts,  one 
which  illustrated  a  typical  display  and  described  the  display  matrix  nota¬ 
tion,  one  which  showed  the  logic  flow  diagram  symbols,  two  which  showed 
how  the  logic  symbols  were  combined  into  flow  diagrams  leading  from 
control  action  to  control  action,  and  one  which  related  the  sequence  of  dis¬ 
plays  diagram  to  action  sequences. 

Following  the  introductory  instructions, a  series  of  teaching  frames 
were  presented.  Each  frame  displayed  a  display  situation  and  diagrammed 
the  acceptable  alternative  action  sequences.  The  taped  instructions  for 
each  frame  described  what  the  display  signified,  what  actions  to  take,  what 
the  actions  did,  and  what  feedback  the  subject  would  receive  if  he  took 
the  correct  actions.  Then  the  subject  was  instructed  to  trace  through 
the  series  of  actions  designated  by  the  logic  flow  diagram.  Taped  instruc¬ 
tions  for  a  sample  frame  are  reproduced  below. 
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Frame  1 


Now  you  will  start  learning  to  use  the  console  during  an  intercept 
mission.  A  diagram  of  the  display  lights  is  projected  on  the  upper  screen. 
Study  the  configuration  of  lights  in  the  diagram,  then  find  where  this  dia¬ 
gram  appears  in  the  sequence  of  displays  which  is  located  on  the  middle 
panel.  Reference  to  the  sequence  of  displays  helps  you  to  follow  the  mis¬ 
sion.  When  you  have  found  the  matching  display  in  the  sequence  of  dis  - 
plays,  look  at  the  label  under  the  diagram,  it  should  say  ACKNOWLEDGE 
ASSIGNMENT.  Now  look  at  the  lower  projection  screen.  You  will  see  a 
diagram  connecting  various  pushbuttons.  This  logic  diagram  identifies 
the  number  and  order  of  pushbuttons  which  are  appropriate  for  this  dis¬ 
play. 

You  are  required  to  accept  and  to  follow  the  assigned  mission.  Four 
lights  in  row  4  of  the  display  (lights  4B,  4C,  4C,  and  4E)  will  go  off  when 
you  have  acknowledged  assignment. 

You  have  to  take  the  following  actions  to  acknowledge  assignment: 

First  Step  -  prepare  the  computer  by  pressing  the  four  "Clear"  but¬ 
tons  which  are  designated  by  the  unordered  "and"  symbol  at  the  far  left 
of  the  projected  diagrams. 

Second  Step  -  activate  display  information  mode  by  pressing  the 
"Display  Information"  button  (designated  by  the  next  ordered  "and"  sym¬ 
bol). 


Third  Step  -  acknowledge  assignment  by  pushing  the  "Acknowledge 
Assignment"  button  (next  on  the  diagram). 

Fourth  Step  -  Activate  computer.  Here  you  have  to  choose  between 
two  equally  appropriate  branches.  If  you  select  the  upper  branch,  you 
will  have  to  enter  the  track  number  by  pushing  four  "Track  Number"  but¬ 
tons  (this  stores  the  identification  of  the  aircraft  track  you  are  following), 
then  activate  the  computer  by  pushing  either  of  the  two  "Activate  Action" 
buttons.  The  computer  will  then  perform  the  programed  steps  and  update 
the  display.  If,  on  the  other  hand,  you  choose  the  lower  branch,  after  the 
acknowledge  assignment  button  you  use  the  "Light  Gun"  button,  which  also 
causes  the  computer  to  store  the  identification  of  the  designated  aircraft 
track.  In  this  sequence  the  same  results  are  obtained  by  using  the  light 
gun  button,  or  by  using  the  track  number  and  activate  action  buttons  (both 
methods  store  the  track  number  and  activate  the  computer). 
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Now,  start  at  the  left  and  push  the  buttons  indicated  by  the  logic  dia¬ 
gram.  First,  push  the  4  clear  buttons - the  order  is  not  important. 

Next,  push  the  display  information  button,  then  the  acknowledge  assign¬ 
ment  button.  Now  you  are  at  a  choice  point,  select  either  the  upper  path 
or  the  lower  path.  Do  not  follow  both  paths.  If  you  select  the  lower  path, 

simply  push  the  light  gun  button - then  you  are  finished.  If  you  select 

the  upper  path  push  the  4  track  number  buttons  (in  order)  aid  either  of  the 
activate  buttons. 

If  during  a  real  mission  you  follow  these  steps  correctly,  the  four 
lights  in  the  fourth  row  will  go  off. 

If  during  a  real  mission  you  had  followed  these  steps  the  display  would 
not  change,  and  you  would  start  again  on  the  left  and  try  to  do  everything 
correctly.  Unless  you  wish  to  review  this  slide  again,  we  will  go  on  to 
the  next  display. 

Twelve  frames  were  presented,  covering  each  alternative  mission 
sequence.  Then  all  the  sequences  were  reviewed.  During  the  review  of 
a  frame  the  display  was  projected  on  the  upper  panel  but  was  not  inter¬ 
preted  for  the  subject.  He  was  instructed  to  compare  it  with  those  shown 
in  the  sequence  of  displays  diagram,  and  then  take  the  appropriate  actions. 
After  a  short  interval  he  was  told  the  display  designation  and  the  list  of 
appropriate  actions  was  read.  He  was  told  to  trace  through  these  actions 
again  as  the  list  was  repeated. 


APPENDIX  F 


Instruction  Manual  for  SAGE  Console  Task 

The  instruction  manual  covered  the  same  information  as  did  the 
slide /tape  presentation.  With  the  exception  of  minor  wording  changes, 
the  organization  and  figures  were  exact  duplicates.  The  same  number 
and  type  of  frames  were  presented,  a  sample  of  which  appears  below. 

Display  Number  1 


Acknowledge  Assignment 


A  B  C  D  E 


o 

o 

o 

# 

• 

• 

o 

o 

• 

• 

o 

o 

o 

• 

• 

o 

• 

• 

• 

• 

o 

o 

o 

o 

o 

Problem:  Accept  an  assigned  mission,  direct  computer  to  follow  a  parti¬ 
cular  aircraft  radar  track. 

Required  Actions: 


1.  Clear  computer 

2.  Activate  display  information  mode 

3.  Acknowledge  assignment 

4.  Insert  track  number  (aircraft  identifying  number) 

5.  Activate  computer 


Action  Sequence: 


act. 

actn. 

act. 

actn. 


light  gun 
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Display  Change: 

When  the  operator  properly  accepts  a  mission,  all  lights  in  row  4 
(4B,  4C,  4D,  and  4E)  go  off.  Other  lights  remain  as  before. 

Explanation  of  Operation  Actions: 

Problem:  Acknowledge  Assignment 


Purpose:  To  acknowledge 

a  mission  assignment 

Action 

Purpose 

Result 

1. 

Push  CLEAR  buttons 

To  clear  the  computer 

(four) 

for  storage  of  new  info 

2. 

Push  DISPLAY 

To  program  computer 

Computer  program 

INFORMATION 

to  display  info  on  the 

remains  inactive 

button 

screen 

3. 

Push  ACKNOWLEDGE 

To  store  info  for  the 

Stored  info  remains 

ASSIGNMENT  button 

computer  program 

inactive 

4. 

Select  option  a  or  b 
below: 

a.  push  TRACK  NO. 

To  store  Track  No.  of 

Stored  Track  No. 

buttons  (four) 

the  assigned  aircraft 
for  the  computer  pro¬ 

remains  inactive 

AND 

gram 

either  of  the 

To  activate  computer 

Computer  performs 

ACTIVATE 

program  to  handle  the 

the  programed  steps 

ACTION  buttons 

stored  info 

and  updates  the  dis¬ 
play 

OR 

b.  push  LIGHT  GUN 

To  use  the  Light  Gun 

Same  as  above 

button 

to  store  Track  No.  and 
activate  the  computer 

The  manual  was  31  pages  in  length.  On  the  last  page,  the  reader  is 
referred  back  to  the  first  frame  to  review  the  sequences  a  second  time. 
Since  the  reader  was  seated  at  the  console  while  studying  the  manual,  he 
was  also  told  to  observe  the  various  elements  of  the  console  and  relate 
these  to  the  text  material. 
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APPENDIX  G 


Instruction  Manual  for  the  Numbers  Game 

The  complete  instruction  manual  is  reproduced  in  the  following  para¬ 
graphs.  Note  that  the  manual  is  organized  to  emphasize  the  decision 
points  and  alternative  actions  shown  in  the  over -all  task  diagram  for  the 
game  (Figure  4). 

Instructions 


1.  General 


In  this  game  you  have  to  determine  a  four  digit  "unknown"  number 
through  a  process  of  logical  deduction.  On  the  sheet  before  you  is  a  list 
of  four  digit  test  numbers  (this  list  is  called  the  LTN)  which  you  will  use 
to  obtain  information  about  the  "unknown".  You  will  select  a  test  number 
from  the  LTN,  write  the  code  identification  for  this  number  on  a  3  x  5 
card,  and  place  the  card  in  the  window  of  the  partition  beside  you.  A 
"score"  will  then  be  written  on  the  card,  and  this  information  displayed 
to  you.  The  score  will  tell  you  how  many  of  the  digits  in  your  test  num¬ 
ber  correspond  (in  both  value  and  position)  to  digits  in  the  "unknown"  num¬ 
ber.  By  keeping  track  of  the  information  learned  from  each  score,  you 
will  finally  discover  the  digits  of  the  "unknown".  To  help  you  keep  track 
of  the  score  information,  there  is  a  Possible  Digits  Register  (PDR)  and 
a  Scores  Register  (SR)  on  the  sheet  of  paper  with  the  LTN.  The  instruc¬ 
tions  will  describe  how  to  use  these  aids. 

2.  How  to  Select  and  Submit  a  Test  Number 

For  your  first  try,  you  may  select  any  of  the  test  numbers.  Note 
the  code  identification  for  the  test  number --write  this  code  on  a  3  x  5 
card  and  pass  it  through  the  window.  After  the  experimenter  writes  the 
score  on  the  card,  he  will  show  the  card  to  you --after  you  have  had  the 
opportunity  to  see  and  record  the  score,  the  experimenter  will  keep  the 
card.  The  score  tells  you  specific  bits  of  positive  and/or  negative  infor¬ 
mation  about  the  "unknown"  number  as  well  as  directing  your  choice  of 
subsequent  test  numbers.  For  example: 


If  you  select  the  first  test  number  (3102),  write  #1  (the  code) 
on  the  card.  If  the  unknown  number  were  4603,  the  score 
would  be  1  since  only  the  third  digits  (0)  are  identical  in  the 
two  numbers.  If  you  then  select  the  12th  test  number  (3251), 
write  #12  on  the  card.  The  score  will  be  0  since  none  of  the 
digits  correspond  in  both  value  and  position  with  any  in  the 
unknown.  Because  you  received  a  score  of  1  on  the  first  test 
number,  you  knew  that  one  and  only  one  of  the  digits  was  cor¬ 
rect,  but  you  did  not  know  which  one.  Consequently,  the  12th 
test  number  was  selected  to  learn  if  the  "3"  in  the  first  col¬ 
umn  was  the  correct  digit.  Thus,  you  follow  up  previously 
scored  numbers  by  trying  other  numbers  which  have  a  single 
corresponding  digit.  If  you  had  received  a  zero  score  for 
3102,  you  would  not  select  3251  as  the  second  try.  In  this 
case,  you  would  already  know  that  the  first  digit  cannot  be  3, 
so  you  would  select  a  test  number  in  which  all  four  digits  re¬ 
main  as  "possibilities".  Use  test  numbers  which  will  provide 
the  most  information  on  each  try. 

3.  How  to  Enter  Zero  Scores  in  the  PDR 

The  Possible  Digits  Register  (PDR)  will  help  you  keep  track  of  which 
digits  you  rule  out  for  each  position  in  the  "unknown"  number.  The  "un¬ 
known"  may  be  regarded  as  comprising  four  columns  in  each  of  which  one 
of  the  digits  from  1-9  or  zero  may  appear.  When  you  receive  a  zero  score 
on  a  test  number,  the  number  of  possible  digits  which  could  appear  in  each 
of  the  four  columns  is  reduced.  To  keep  track  of  those  digits  which  have 
been  ruled  out,  X  out  the  corresponding  digits  in  the  PDR.  The  four  col¬ 
umns  of  the  PDR  correspond  to  the  four  digits  of  the  test  numbers,  the  10 
rows  of  the  PDR  correspond  to  the  numerical  value  of  each  digit  (1-9  or 
zero)  which  can  appear  in  any  column.  For  example: 

If  your  first  test  number  is  #12  (3251)  and  you  receive  a  score 
of  zero--X  out  3  in  the  1st  column,  2  in  the  2nd,  5  in  the  3rd, 
and  1  in  the  4th.  The  PDR  would  then  look  like  this: 


1 

2 

3 

4 

5 

6 

7 

8 
9 
0 


Columns 


X 

X 

X 

X 
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4.  How  to  Enter  Scores  of  1  or  2  in  the  SR 


The  Scores  Register  (SR)  will  help  you  keep  track  of  which  digits 
probably  appear  in  the  "unknown"  and  should  be  followed  up.  When  you 
receive  a  score  of  1  or  2,  simply  write  the  complete  test  number  in  a 
row  of  the  SR  and  the  score  in  the  same  row  of  the  score  column.  For 
example: 

If  your  first  test  number  is  #1  (3102)  and  you  receive  a  score 
of  1,  enter  these  in  the  SR.  The  SR  would  then  look  like  this: 

Columns 


Test  Num 

Der 

Score 

3 

1 

0 

2 

1 

5.  How  to  Update  the  PDR  and  SR 

After  entering  the  results  of  your  first  test  number  in  either  the  PDR 
or  SR,  select  a  second  test  number  to  follow  up  SR  cores  or  to  rule  out 
further  possibilities  in  the  PDR.  If  subsequent  test  numbers  receive  a 
score  of  zero,  X  out  the  possibilities  in  the  PDR,  then  check  previous  en¬ 
tries  in  the  SR  to  see  if  any  of  the  digits  correspond  to  possibilities  which 
have  now  been  ruled  out.  If  any  have  been  ruled  out,  you  should  X  them 
out  in  the  SR  also.  Then,  check  to  see  if  the  remaining  number  of  digits 
in  any  row  of  the  SR  equals  the  score  for  that  row.  If  they  do,  then  you 
have  identified  digits  in  the  "unknown".  For  example: 

If  your  first  number  is  #1  (3102)  and  is  scored  2,  your  second 
test  number  is  #12  (32  51)  and  is  scored  zero,  your  third  test 
number  is  #17  (7123)  and  is  scored  1,  and  your  fourth  test 
number  is  #6  (7892)  and  is  scored  zero--you  would  have  crossed 
out  the  first  and  last  digits  of  3102,  leaving  the  number  of  re¬ 
maining  digits  equal  to  the  score  (2).  The  PDR  and  SR  would 
then  look  like  this: 
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PDR 


Columns 


1 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

0 

SR 


Columns 
Test  Number 


3 

1 

0 

2 

3 

2 

5 

1 

7 

1 

2 

3 

7 

8 

9 

2 

Score 

2 

0 

1 

0 


Circle  positively  identified  digits  in  the  SR  (intiiis  case  1  in  the 
2nd  column,  and  0  in  the  3rd  column),  enter  the  identified  digits 
in  the  PDR  and  X  out  possibilities  which  have  been  ruled  out. 
Since  you  have  identified  the  digits  in  the  2nd  and  3rd  columns, 

X  out  all  the  rest  of  the  spaces  in  these  columns.  The  remain¬ 
ing  entries  in  the  SR  are  then  rechecked  to  X  out  digits  in  the 
2nd  and  3rd  columns  which  had  been  crossed  out  in  the  PDR. 
Subsequent  test  numbers  would  then  be  selected  to  include  digits 
in  the  first  and  fourth  columns  which  have  not  yet  been  ruled  out. 


Continue  to  evaluate  additional  test  numbers  until  you  can  deduce  the 
complete  "unknown"  four  digit  number.  When  you  know  the  complete  num¬ 
ber,  write  it  on  a  card  and  pass  it  through  the  window  to  the  experimenter. 
He  will  tell  you  if  you  are  correct  or  write  "No"  on  the  card  if  you  are 
wrong.  Continue  to  play  until  you  are  correct. 

You  will  play  the  game  a  predetermined  number  of  times,  or  until 
time  for  the  test  session  runs  out.  The  object  of  the  game  is  to  work  as 
quickly  as  possible  to  learn  the  "unknowns"  with  as  few  test  numbers  as 
possible.  You  may  commence  as  soon  as  you  feel  that  you  understand 
how  to  proceed. 


i 
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APPENDIX  H 


Performance  Measures  for  the  Numbers  Game 


All  time  measures  are  in  minutes,  other  measures  are  number  of 


queries.  The  submission  of  the  correct  four  digit  number  (which  termi¬ 
nated  the  game)  is  not  included  in  the  totals.  Incorrect  guesses  for  the 
"unknown"  four  digit  number  were  always  categorized  as  "no  value". 
Subjects  1  through  6  read  the  instruction  manual,  and  7  through  12  used 


the  logic  flow  diagrams. 


Task 


Training 


Time 

A 

B 

C 

D 

E 

F 

Total  Requests 
for  Each  Subject 

1 

10.  0 

18 

13 

9 

12 

24 

15 

2 

13.  4 

17 

12 

13 

15 

13 

14 

3 

8.4 

18 

14 

12 

14 

16 

14 

4 

8.  0 

13 

21 

19 

14 

11 

11 

5 

11.0 

14 

17 

19 

14 

10 

15 

6 

12.  5 

12 

❖ 

20 

27 

14 

15 

7 

53.  0 

15 

16 

10 

❖ 

18 

20 

8 

5.0 

12 

13 

11 

12 

11 

15 

9 

4.  0 

13 

10 

18 

20 

14 

12 

10 

17.  0 

17 

10 

13 

12 

11 

13 

11 

12. 0 

15 

12 

15 

13 

13 

13 

12 

24.  0 

14 

12 

17 

12 

12 

12 

Total  High  Value 

Queries 

1 

9 

10 

8 

8 

9 

14 

2 

14 

12 

10 

12 

11 

13 

3 

7 

11 

8 

6 

9 

8 

4 

11 

9 

12 

12 

12 

11 

5 

8 

13 

10 

8 

7 

12 

6 

9 

* 

9 

12 

9 

9 

7 

12 

12 

9 

❖ 

9 

8 

8 

9 

9 

10 

8 

10 

11 

9 

9 

7 

12 

9 

14 

11 

10 

10 

10 

10 

8 

8 

12 

11 

11 

9 

12 

12 

10 

10 

12 

11 

9 

11 

12 

12 

12 

-  73  - 
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Task 

• 

Total  Low  Value 
Queries 

A 

B 

C 

D 

E 

F 

1 

3 

2 

1 

3 

5 

1 

2 

3 

0 

3 

1 

2 

1 

3 

3 

2 

2 

3 

3 

4 

4 

1 

0 

2 

2 

0 

0 

t 

5 

3 

0 

4 

5 

2 

1 

• 

6 

0 

a# 

V 

2 

0 

3 

0 

V 

7 

3 

1 

1 

sU 

4 

6 

1 

8 

3 

1 

1 

2 

1 

3 

4 

9 

3 

3 

3 

3 

0 

0 

10 

2 

0 

2 

2 

3 

1 

11 

3 

3 

1 

1 

3 

3 

12 

Total  No  Value 

Queries 

1 

3 

0 

0 

0 

0 

1 

6 

1 

0 

1 

10 

0 

2 

0 

0 

0 

2 

0 

0 

3 

8 

1 

2 

1 

4 

2 

t 

4 

1 

12 

5 

0 

0 

0 

5 

3 

4 

5 

1 

1 

2 

6 

3 

* 

9 

15 

2 

6 

• 

7 

0 

3 

0 

4. 

5 

7 

A 

8 

0 

3 

0 

2 

0 

1 

9 

1 

0 

3 

8 

0 

1 

4 

10 

5 

0 

1 

2 

0 

0 

11 

1 

0 

2 

0 

0 

\ 

\ 

o  o 

12 

Total  Time  to 
Completion 

2 

0 

6 

0 

0 

1 

24.  0 

12.  1 

9.  5 

10.  6 

29.  5 

12.  V 

2 

22.  5 

11.2 

12.  1 

13.  4 

10.  8 

J '  .  8 

3 

24.  0 

11.  6 

9.  0 

7.  9 

10.  0 

7.  1 

4 

10.  6 

24.  6 

17.  0 

8.  1 

6./ 

5.2 

5 

13.  9 

12.  2 

15.  4 

9.  5 

6.8 

9.  9 

6 

8.2 

* 

12.  0 

19.3 

8.  1 

10.  8 

7 

20.  0 

26.  8 

14.  0 

* 

15.0 

15.  3 

4 

8 

9.  7 

9.  2 

6.  3 

7.2 

4.  3 

9.  5 

9 

13.  5 

8.  5 

30.  0 

31. 3 

13.  3 

13.  5 

f 

* 

10 

21.0 

8.  6 

13.  5 

19.  3 

8.  1 

9.  5 

11 

14.  9 

8.  8 

14.  9 

9.  7 

7.  9 

11.0 

• 

12 

47.  0 

24.  9 

33.  6 

16.  0 

15.  5 

11.  5 

*Game  was  discontinued  after  35  minutes  and  2  5  queries. 
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